Is there a means for cluster wide caching

[Legacy User]

Hi Folks,

            The default caching feature is not cluster wide & hence we want to mimic this using the gateway internal DB. We want to store negligible amount of information. Is this a bad practice & going to affect gateway operations?

 

Thanks,

Abhishek

[Stephen Hughes]

Abhishek,

 

Good evening.  We have a tactical assertion that can be provided which allows for remote cache to be enabled   by linking into memcache. Another possiblity depending on the frequency and size of the updates, is using the restman service to update a cluster property which is then shared across all nodes in the cluster.

 

Sincerely,

 

Stephen Hughes

Director, CA Support

[Legacy User]

Hi Stephen,

                   Thanks for the quick response. I was able to understand the cluster property option. Can you please give more details on memcache linking. Is there a documentation that explains this options?

 

Thanks,

Abhishek

[Stephen Hughes]

Abhishek,

 

This will need to be requested through a support case for tracking.

 

Sincerely,

 

Stephen Hughes

Director, CA Support

[Legacy User]

Hi Steve,

        I was trying to use Restman to do a POC for this solution.  I was able to install & play around with the APIs. I am running into issue when I am trying to add a property. Please let me know if I am missing something?

Example-

URL	Request Payload	Response	Audit Log error
PUT https://IP:port/restman/1.0/clusterProperties/sample HTTP/1.1 Accept-Encoding: gzip,deflate Content-Type: application/xml Content-Length: 666 Host: servicesdev.na.cargill.com:8443 Connection: Keep-Alive User-Agent: Apache-HttpClient/4.1.1 (java 1.5) Authorization: Basic ###################	<?xml version="1.0" encoding="UTF-8" standalone="yes"?> <l7:Item xmlns:l7="http://ns.l7tech.com/2010/04/gateway-management">     <l7:Name>CLUSTER_PROPERTY Template</l7:Name>     <l7:Type>CLUSTER_PROPERTY</l7:Type>     <l7:TimeStamp>2016-07-11T06:08:38.776-05:00</l7:TimeStamp>     <l7:Link rel="self" uri="https://soal7d1.cargill.com:8443/restman/1.0/clusterProperties/template"/>     <l7:Link rel="list" uri="https://soal7d1.cargill.com:8443/restman/1.0/clusterProperties"/>     <l7:Resource>         <l7:ClusterProperty>             <l7:Name>sample</l7:Name>             <l7:Value>==2w2ininini</l7:Value>         </l7:ClusterProperty>     </l7:Resource> </l7:Item>	HTTP/1.1 500 Internal Server Error Server: Apache-Coyote/1.1 L7-Policy-URL: https://IP:port/ssg/policy/disco?serviceoid=1f6ef038325871464fc74606d2b805a6 Content-Type: text/xml;charset=utf-8 Content-Length: 44 Date: Mon, 11 Jul 2016 11:31:20 GMT Connection: close     Service Unavailable.  Please try again later	20160711 16:52:38.152 WARNING 9050 Error processing management request: Exception encountered processing a rest message: javax.ws.rs.ProcessingException: Resource Java method invocation error. Caused by: argument type mismatch

 

 

Thanks,

Abhishek

[Stephen Hughes]

Abhishek,

 

Please try the following:

HTTP Method: POST

HTTP URI: /restman/1.0/clusterProperties

Payload:

<l7:ClusterProperty xmlns:l7="http://ns.l7tech.com/2010/04/gateway-management">

            <l7:Name>sample</l7:Name>

            <l7:Value>==2w2ininini</l7:Value>

</l7:ClusterProperty>

 

Any update will require the PUT HTTP Method along with the ID for the cluster property.

Sincerely,

Stephen Hughes

Director, CA Support

[Legacy User]

Thanks Steve. We tried this but ran into another issue. I am getting the below response -

 

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>

<l7:Error xmlns:l7="http://ns.l7tech.com/2010/04/gateway-management">

    <l7:Type>PermissionDenied</l7:Type>

    <l7:TimeStamp>2016-07-13T10:37:46.547-05:00</l7:TimeStamp>

    <l7:Link rel="self" uri="https://soal7d1.cargill.com:8443/restman/1.0/clusterProperties/sample"/>

    <l7:Detail>Permission denied for user #####. Caused by: Permission denied: Create CLUSTER_PROPERTY</l7:Detail>

</l7:Error>

 

I have admin role & I am not sure why this error is returned?

 

Also, I have couple of more questions -

1) What is the max size of cluster property value?

2) We have multiple gateway nodes. By following this approach, is there no probability of Race condition or dirty updates by multiple nodes & requests in same node?

 

 

Thanks,

Abhishek

[Stephen Hughes]

Abhishek,

 

I'm not sure why you received this error message. Please review the restman policy to determine how the security is setup but as long as you are in the administrator role and provide username/password or client certificate you are good.

 

As for your questions:

1) The value can be 16 MB in size for mediumtext in MySQL (

MySQL :: MySQL 5.7 Reference Manual :: 12.8 Data Type Storage Requirements ) I would not recommend using this much room as this has to be loaded into the cache at regular intervals

2) This option is not recommend for high concurrency/multiple changes but for sharing of data between nodes. If you are looking at high volume of changes then a remote cache is a better bet.

 

Sincerely,

Stephen Hughes

Director, CA Support

[Legacy User]

I created a encapsulated assertion to create/update a cluster property. The user must have the rights to read, create and update cluster properties. Make a role for this right. If you want to prevent the user from changing all cluster properties: use a prefix for all the cluster properties you want to be changed and modify the role to use read, create and update cluster properties that starts with the choosen prefix.

 

I do know how to update files, so here is the export of the encapsulated assertion (not a policy export).

 

<?xml version="1.0" encoding="UTF-8"?>
<exp:Export Version="3.0"
    xmlns:L7p="http://www.layer7tech.com/ws/policy"
    xmlns:exp="http://www.layer7tech.com/ws/policy/export" xmlns:wsp="http://schemas.xmlsoap.org/ws/2002/12/policy">
    <exp:References/>
    <wsp:Policy xmlns:L7p="http://www.layer7tech.com/ws/policy" xmlns:wsp="http://schemas.xmlsoap.org/ws/2002/12/policy">
        <wsp:All wsp:Usage="Required">
            <L7p:AuditDetailAssertion>
                <L7p:Detail stringValueReference="inline"><![CDATA[--- Update Cluster Property
Property.Name: ${property.name}
Property.Value: ${property.value}
UserName: ${username}
Password: *******]]></L7p:Detail>
            </L7p:AuditDetailAssertion>
            <L7p:SetVariable>
                <L7p:Base64Expression stringValue="aHR0cHM6Ly9sb2NhbGhvc3Q6ODQ0My9yZXN0bWFuLzEuMA=="/>
                <L7p:VariableToSet stringValue="restmanurl"/>
            </L7p:SetVariable>
            <wsp:OneOrMore wsp:Usage="Required">
                <wsp:All wsp:Usage="Required">
                    <L7p:SetVariable>
                        <L7p:Base64Expression stringValue=""/>
                        <L7p:ContentType stringValue="text/xml; charset=utf-8"/>
                        <L7p:DataType variableDataType="message"/>
                        <L7p:VariableToSet stringValue="allProperties"/>
                    </L7p:SetVariable>
                    <L7p:HttpRoutingAssertion>
                        <L7p:HttpMethod httpMethod="GET"/>
                        <L7p:Login stringValue="${username}"/>
                        <L7p:Password stringValue="${password}"/>
                        <L7p:ProtectedServiceUrl stringValue="${restmanurl}/clusterProperties"/>
                        <L7p:ProxyPassword stringValueNull="null"/>
                        <L7p:ProxyUsername stringValueNull="null"/>
                        <L7p:RequestHeaderRules httpPassthroughRuleSet="included">
                            <L7p:Rules httpPassthroughRules="included"/>
                        </L7p:RequestHeaderRules>
                        <L7p:RequestParamRules httpPassthroughRuleSet="included">
                            <L7p:ForwardAll booleanValue="true"/>
                            <L7p:Rules httpPassthroughRules="included"/>
                        </L7p:RequestParamRules>
                        <L7p:ResponseHeaderRules httpPassthroughRuleSet="included">
                            <L7p:ForwardAll booleanValue="true"/>
                            <L7p:Rules httpPassthroughRules="included">
                                <L7p:item httpPassthroughRule="included">
                                    <L7p:Name stringValue="Set-Cookie"/>
                                </L7p:item>
                            </L7p:Rules>
                        </L7p:ResponseHeaderRules>
                        <L7p:ResponseMsgDest stringValue="allProperties"/>
                        <L7p:SamlAssertionVersion intValue="2"/>
                    </L7p:HttpRoutingAssertion>
                    <L7p:AuditDetailAssertion>
                        <L7p:Detail stringValueReference="inline"><![CDATA[All Properties
${allProperties.mainpart}]]></L7p:Detail>
                    </L7p:AuditDetailAssertion>
                    <L7p:SetVariable>
                        <L7p:Base64Expression stringValue="L2w3Okxpc3QvbDc6SXRlbVtsNzpOYW1lPScke3Byb3BlcnR5Lm5hbWV9J10vbDc6TGluay9AdXJp"/>
                        <L7p:VariableToSet stringValue="xpath"/>
                    </L7p:SetVariable>
                    <L7p:ResponseXpathAssertion>
                        <L7p:AssertionComment assertionComment="included">
                            <L7p:Properties mapValue="included">
                                <L7p:entry>
                                    <L7p:key stringValue="RIGHT.COMMENT"/>
                                    <L7p:value stringValue="-- Extract current value"/>
                                </L7p:entry>
                            </L7p:Properties>
                        </L7p:AssertionComment>
                        <L7p:VariablePrefix stringValue="uri"/>
                        <L7p:XmlMsgSrc stringValue="allProperties"/>
                        <L7p:XpathExpression xpathExpressionValue="included">
                            <L7p:Expression stringValue="${xpath}"/>
                            <L7p:Namespaces mapValue="included">
                                <L7p:entry>
                                    <L7p:key stringValue="s"/>
                                    <L7p:value stringValue="http://schemas.xmlsoap.org/soap/envelope/"/>
                                </L7p:entry>
                                <L7p:entry>
                                    <L7p:key stringValue="l7"/>
                                    <L7p:value stringValue="http://ns.l7tech.com/2010/04/gateway-management"/>
                                </L7p:entry>
                            </L7p:Namespaces>
                            <L7p:XpathVersion xpathVersion="XPATH_1_0"/>
                        </L7p:XpathExpression>
                    </L7p:ResponseXpathAssertion>
                    <L7p:SetVariable>
                        <L7p:Base64Expression stringValue="JHt1cmkucmVzdWx0fQ=="/>
                        <L7p:VariableToSet stringValue="prop.uri"/>
                    </L7p:SetVariable>
                    <L7p:AuditDetailAssertion>
                        <L7p:Detail stringValue="Property exists on ${prop.uri}"/>
                    </L7p:AuditDetailAssertion>
                    <L7p:assertionComment>
                        <L7p:Properties mapValue="included">
                            <L7p:entry>
                                <L7p:key stringValue="LEFT.COMMENT"/>
                                <L7p:value stringValue="Property exists >>"/>
                            </L7p:entry>
                        </L7p:Properties>
                    </L7p:assertionComment>
                </wsp:All>
                <wsp:All wsp:Usage="Required">
                    <L7p:SetVariable>
                        <L7p:Base64Expression stringValue=""/>
                        <L7p:VariableToSet stringValue="prop.uri"/>
                    </L7p:SetVariable>
                    <L7p:assertionComment>
                        <L7p:Properties mapValue="included">
                            <L7p:entry>
                                <L7p:key stringValue="LEFT.COMMENT"/>
                                <L7p:value stringValue="New Property >>"/>
                            </L7p:entry>
                        </L7p:Properties>
                    </L7p:assertionComment>
                </wsp:All>
                <L7p:assertionComment>
                    <L7p:Properties mapValue="included">
                        <L7p:entry>
                            <L7p:key stringValue="RIGHT.COMMENT"/>
                            <L7p:value stringValue="-- Check if there is a current property"/>
                        </L7p:entry>
                    </L7p:Properties>
                </L7p:assertionComment>
            </wsp:OneOrMore>
            <L7p:SetVariable>
                <L7p:Base64Expression stringValue="PGw3OkNsdXN0ZXJQcm9wZXJ0eSB4bWxuczpsNz0iaHR0cDovL25zLmw3dGVjaC5jb20vMjAxMC8wNC9nYXRld2F5LW1hbmFnZW1lbnQiPg0KIDxsNzpOYW1lPiR7cHJvcGVydHkubmFtZX08L2w3Ok5hbWU+DQogPGw3OlZhbHVlPiR7cHJvcGVydHkudmFsdWV9PC9sNzpWYWx1ZT4NCjwvbDc6Q2x1c3RlclByb3BlcnR5Pg=="/>
                <L7p:ContentType stringValue="text/xml; charset=utf-8"/>
                <L7p:DataType variableDataType="message"/>
                <L7p:VariableToSet stringValue="payload"/>
            </L7p:SetVariable>
            <wsp:OneOrMore wsp:Usage="Required">
                <wsp:All wsp:Usage="Required">
                    <L7p:ComparisonAssertion>
                        <L7p:CaseSensitive booleanValue="false"/>
                        <L7p:Expression1 stringValue="${prop.uri}"/>
                        <L7p:MultivaluedComparison multivaluedComparison="FAIL"/>
                        <L7p:Operator operatorNull="null"/>
                        <L7p:Predicates predicates="included">
                            <L7p:item dataType="included">
                                <L7p:Type variableDataType="string"/>
                            </L7p:item>
                            <L7p:item stringLength="included">
                                <L7p:Max intValue="-1"/>
                                <L7p:Min intValue="1"/>
                            </L7p:item>
                        </L7p:Predicates>
                    </L7p:ComparisonAssertion>
                    <L7p:HttpRoutingAssertion>
                        <L7p:HttpMethod httpMethod="PUT"/>
                        <L7p:Login stringValue="${username}"/>
                        <L7p:Password stringValue="${password}"/>
                        <L7p:ProtectedServiceUrl stringValue="${prop.uri}"/>
                        <L7p:ProxyPassword stringValueNull="null"/>
                        <L7p:ProxyUsername stringValueNull="null"/>
                        <L7p:RequestHeaderRules httpPassthroughRuleSet="included">
                            <L7p:Rules httpPassthroughRules="included"/>
                        </L7p:RequestHeaderRules>
                        <L7p:RequestMsgSrc stringValue="payload"/>
                        <L7p:RequestParamRules httpPassthroughRuleSet="included">
                            <L7p:ForwardAll booleanValue="true"/>
                            <L7p:Rules httpPassthroughRules="included"/>
                        </L7p:RequestParamRules>
                        <L7p:ResponseHeaderRules httpPassthroughRuleSet="included">
                            <L7p:ForwardAll booleanValue="true"/>
                            <L7p:Rules httpPassthroughRules="included">
                                <L7p:item httpPassthroughRule="included">
                                    <L7p:Name stringValue="Set-Cookie"/>
                                </L7p:item>
                            </L7p:Rules>
                        </L7p:ResponseHeaderRules>
                        <L7p:SamlAssertionVersion intValue="2"/>
                    </L7p:HttpRoutingAssertion>
                    <L7p:assertionComment>
                        <L7p:Properties mapValue="included">
                            <L7p:entry>
                                <L7p:key stringValue="LEFT.COMMENT"/>
                                <L7p:value stringValue="Update >>"/>
                            </L7p:entry>
                        </L7p:Properties>
                    </L7p:assertionComment>
                </wsp:All>
                <wsp:All wsp:Usage="Required">
                    <L7p:ComparisonAssertion>
                        <L7p:CaseSensitive booleanValue="false"/>
                        <L7p:Expression1 stringValue="${prop.uri}"/>
                        <L7p:ExpressionIsVariable booleanValue="false"/>
                        <L7p:MultivaluedComparison multivaluedComparison="FAIL"/>
                        <L7p:Operator operatorNull="null"/>
                        <L7p:Predicates predicates="included">
                            <L7p:item dataType="included">
                                <L7p:Type variableDataType="string"/>
                            </L7p:item>
                            <L7p:item stringLength="included"/>
                        </L7p:Predicates>
                    </L7p:ComparisonAssertion>
                    <L7p:HttpRoutingAssertion>
                        <L7p:HttpMethod httpMethod="POST"/>
                        <L7p:Login stringValue="${username}"/>
                        <L7p:Password stringValue="${password}"/>
                        <L7p:ProtectedServiceUrl stringValue="${restmanurl}/clusterProperties"/>
                        <L7p:ProxyPassword stringValueNull="null"/>
                        <L7p:ProxyUsername stringValueNull="null"/>
                        <L7p:RequestHeaderRules httpPassthroughRuleSet="included">
                            <L7p:Rules httpPassthroughRules="included"/>
                        </L7p:RequestHeaderRules>
                        <L7p:RequestMsgSrc stringValue="payload"/>
                        <L7p:RequestParamRules httpPassthroughRuleSet="included">
                            <L7p:ForwardAll booleanValue="true"/>
                            <L7p:Rules httpPassthroughRules="included"/>
                        </L7p:RequestParamRules>
                        <L7p:ResponseHeaderRules httpPassthroughRuleSet="included">
                            <L7p:ForwardAll booleanValue="true"/>
                            <L7p:Rules httpPassthroughRules="included">
                                <L7p:item httpPassthroughRule="included">
                                    <L7p:Name stringValue="Set-Cookie"/>
                                </L7p:item>
                            </L7p:Rules>
                        </L7p:ResponseHeaderRules>
                        <L7p:SamlAssertionVersion intValue="2"/>
                    </L7p:HttpRoutingAssertion>
                    <L7p:assertionComment>
                        <L7p:Properties mapValue="included">
                            <L7p:entry>
                                <L7p:key stringValue="LEFT.COMMENT"/>
                                <L7p:value stringValue="Create >>"/>
                            </L7p:entry>
                        </L7p:Properties>
                    </L7p:assertionComment>
                </wsp:All>
                <L7p:assertionComment>
                    <L7p:Properties mapValue="included">
                        <L7p:entry>
                            <L7p:key stringValue="RIGHT.COMMENT"/>
                            <L7p:value stringValue="-- Create or update property"/>
                        </L7p:entry>
                    </L7p:Properties>
                </L7p:assertionComment>
            </wsp:OneOrMore>
            <L7p:SetVariable>
                <L7p:Base64Expression stringValue="JHtyZXNwb25zZS5tYWlucGFydH0="/>
                <L7p:VariableToSet stringValue="result"/>
            </L7p:SetVariable>
            <L7p:AuditDetailAssertion>
                <L7p:Detail stringValueReference="inline"><![CDATA[*** Update Cluster Property
Result: ${result}]]></L7p:Detail>
            </L7p:AuditDetailAssertion>
        </wsp:All>
    </wsp:Policy>
    <enc:EncapsulatedAssertion
        guid="a90b7ef6-2f80-4dfb-8429-7aa6c9334629"
        xmlns:L7="http://ns.l7tech.com/secureSpan/1.0/core"
        xmlns:enc="http://ns.l7tech.com/secureSpan/1.0/encass"
        xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
        <L7:name>Update Cluster Property_20160826</L7:name>
        <enc:Policy guid="f13c30c2-cd01-4ff3-8146-ed2f18c6d12c" name="Update Cluster Property_20160826"/>
        <enc:EncapsulatedAssertionArguments>
            <enc:EncapsulatedAssertionArgument
                id="9aea22e27c30d33be08dbeb292794afa" version="0">
                <enc:ArgumentName>username</enc:ArgumentName>
                <enc:ArgumentType>string</enc:ArgumentType>
                <enc:GuiLabel>Username</enc:GuiLabel>
                <enc:GuiPrompt>true</enc:GuiPrompt>
                <enc:Ordinal>3</enc:Ordinal>
            </enc:EncapsulatedAssertionArgument>
            <enc:EncapsulatedAssertionArgument
                id="9aea22e27c30d33be08dbeb292794afc" version="0">
                <enc:ArgumentName>password</enc:ArgumentName>
                <enc:ArgumentType>string</enc:ArgumentType>
                <enc:GuiLabel>Password</enc:GuiLabel>
                <enc:GuiPrompt>true</enc:GuiPrompt>
                <enc:Ordinal>4</enc:Ordinal>
            </enc:EncapsulatedAssertionArgument>
            <enc:EncapsulatedAssertionArgument
                id="9aea22e27c30d33be08dbeb292794af8" version="0">
                <enc:ArgumentName>property.value</enc:ArgumentName>
                <enc:ArgumentType>string</enc:ArgumentType>
                <enc:GuiLabel>Property Value</enc:GuiLabel>
                <enc:GuiPrompt>true</enc:GuiPrompt>
                <enc:Ordinal>2</enc:Ordinal>
            </enc:EncapsulatedAssertionArgument>
            <enc:EncapsulatedAssertionArgument
                id="9aea22e27c30d33be08dbeb292794af6" version="0">
                <enc:ArgumentName>property.name</enc:ArgumentName>
                <enc:ArgumentType>string</enc:ArgumentType>
                <enc:GuiLabel>Property Name</enc:GuiLabel>
                <enc:GuiPrompt>true</enc:GuiPrompt>
                <enc:Ordinal>1</enc:Ordinal>
            </enc:EncapsulatedAssertionArgument>
        </enc:EncapsulatedAssertionArguments>
        <enc:EncapsulatedAssertionResults>
            <enc:EncapsulatedAssertionResult
                id="9aea22e27c30d33be08dbeb292794afe" version="0">
                <enc:ResultName>result</enc:ResultName>
                <enc:ResultType>string</enc:ResultType>
            </enc:EncapsulatedAssertionResult>
        </enc:EncapsulatedAssertionResults>
        <enc:Properties>
            <entry>
                <key xsi:type="xs:string">paletteFolder</key>
                <value xsi:type="xs:string">audit</value>
            </entry>
            <entry>
                <key xsi:type="xs:string">allowTracing</key>
                <value xsi:type="xs:string">false</value>
            </entry>
            <entry>
                <key xsi:type="xs:string">paletteIconResourceName</key>
                <value xsi:type="xs:string">CreateWSDL16x16.gif</value>
            </entry>
            <entry>
                <key xsi:type="xs:string">artifactVersion</key>
                <value xsi:type="xs:string">9b02dd18cc14ea1138c984d6aae1e3417136ca6b</value>
            </entry>
            <entry>
                <key xsi:type="xs:string">policyGuid</key>
                <value xsi:type="xs:string">f13c30c2-cd01-4ff3-8146-ed2f18c6d12c</value>
            </entry>
        </enc:Properties>
    </enc:EncapsulatedAssertion>
</exp:Export>

[Legacy User]

Memcache is not a real option for storing data. If a node of a two-node cluster is down, you loose half of the data and it will not store half of the data till the whole cluster is up or you change the cluster configuration. Memcache is meant for caching websites. If it is down you can fallback on the original source. Good for webcaching, bad for data. I would also really like storing volatile data which is replicated in the cluster.

[mogol01]

The remote cache assertion supports Memcached, Gemfire, Coherence, Terracotta, and Redis. The characteristics of the caching vendor aside, all Gateway nodes will talk to the same off-box caching server - or cluster of servers - hence why this is also speaks to the use case of general cluster-wide caching.

 

Note that there is also a latency penalty due to the round-trip off-box network jump.

 

And yes, the remote cache assertion is indeed intended to be used for off-box caching - where original source can be retrieved - and not storage.

[Jay Thorne]

In 9.1 the option of using Cassandra is a useful high speed data store. If it needs to be faster than Cassandra, then the on-box cache per node should work fine, if you write an API to fill the data. 

 

What's more important than performance and tools we support is the use case. 

 

What are you trying to do, from a business perspective? Share configuration data?

[Legacy User]

I want to use a distributed cache for storing session information and (authentication) validations. I do not want to use the database for this, because of the performance penalty and the cleanup of the expired sessions/validations. We are now using sticky sessions on the load balancer and use the local cache. I would prefer non-sticky sessions if we can.