Deploying WSO2 products in Google Container Cloud

This blog explains the steps of deploying WSO2 products in Google Container Cloud.


More info can be found in following WSO2 Official documentation.

Kubernetes Artifacts

Dockerfiles Documentation

Puppet Modules

1) Download and Install

Google Cloud SDK 

kubectl 

2) Subscribe

Subscribe to 'Google Cloud Trial'

3) Setup / Configure

i) A container cluster

WHAT : This is the cluster of container hosts.

HOW :

• Go to Google Container Engine 

• Create a container cluster. (Change the machine type and node count if you need)

ii) kubectl 

WHY : kubectl should be configured with Google Container Cloud, so that the kubectl commands can be invoked against the cloud.

HOW : Instructions can be found by clicking on 'connect' on container cluster entry we just created in the previous step.

4) Git Clone

IMPORTANT : Please make sure you checkout a release tag of these repos since the master branch would contain not production ready RnD stuff.

i) https://github.com/wso2/kubernetes-artifacts

WHAT : The repo of services and replication controllers for WSO2 products

ii) https://github.com/wso2/dockerfiles

WHAT : Docker files to build WSO2 product images.

iii) https://github.com/wso2/puppet-modules

WHAT : Puppet scripts to deploy WSO2 products in a host.

WHY : These puppet scripts are used by the docker files to build an image with WSO2 product deployments.

5) Build Images

TWEAK :

WSO2 Kubernetes clustering scheme talks to the Kubernetes API to get node data. Because of the JVM version difference in WSO2 Docker Images (Java 7) and Google VMs (Java 8), we need to add the following line to PUPPET_MODULES_GIT_REPO/modules/<product>/templates/<version>/bin/wso2server.sh.erb  

-Ddeployment.security.SSLv2Hello=false \
-Ddeployment.security.SSLv3=false \
-Ddeployment.security.TLSv1=false \
-Ddeployment.security.TLSv1.1=true \
-Ddeployment.security.TLSv1.2=true \
-Dhttps.protocols=TLSv1.2 \

HOW : Refer to the 'Build Docker Images' section here.

6) Push the Images

HOW : Refer this 

7) Deploy Services and Replication Controllers

 

HOW :

• Go to KUBERNETES_ARTIFACTS_GIT_REPO/<product>

• ./deploy sh

TWEAK :

The image names in the deployed replication controllers are not valid in the Google Container Cloud env. So we have to amend the replication controller definitions to refer to the image names in the Google container registry.

We can do it from the Google Kubernetes Dashboard

Select 'View/edit YAML' of the desired replication controller.

Edit the spec/containers/image element with the correct image name.

e.g. gcr.io/dazzling-bruin-148110/wso2am-kubernetes:1.10.0

In order to make the change effective you can scale the nodes to 0 and then scale up.

8) Open Firewall Ports

WHY : By default the ports of Google cloud nodes are closed for traffic from outside.

HOW : Go to Networking -> Firewall rules section and add a rule to expose the desired ports.

WSO2 App Manager 1.2.0 - How to use custom app properties in ReST APIs

WSO2 App Manager supports defining and using custom properties for app types. In order to add new custom property, the relevant RXT file (registry extension) and couple of other files should be amended. But these custom properties are not marked as 'custom' properties  in any place. Once defined they are treated just like any other another field.  

With the introduction of the new ReST API implementation in App Manager 1.2.0, it was bit of challenging to expose these custom fields through APIs. The new ReST APIs are documented using Swagger. So when the relevant API response models are defined, the custom fields can't be added as named properties since they are dynamic. So the solution is to have a field, which is a Map, to represent the custom properties. And the need of marking custom fields as 'custom' should be addressed too. 

In App Manager, this has been addressed by having another configuration to store the custom properties.

Where is the definitions file

In the registry there are JSON resources which are custom property definitions. There is a definition file per app type.

e.g. Definition file for web apps -  
         /_system/governance/appmgt/applicationdata/custom-property-definitions/webapp.json

What does a definition file look like

As of now the custom property definitions file only has the names of the custom properties.

e.g.
{"customPropertyDefinitions":[{"name":"overview_custom1"}]}

How do I persist these custom properties for an app using the ReST API

The request payload should contain the custom properties as below.

{
  "name": "app1",
  "version": "1.0",
  "isDefaultVersion":true,
  .
  .
  .
  "customProperties":[
    {
       "name":"overview_custom1",
       "value":"custom_property_1"
    }
  ]
}

JWT's role in WSO2 App Manager

WSO2 App Manager is a fully fledged solution for managing and governing applications in an enterprise.

App Manager 1.0.0 supports web apps and mobile apps, out of the box.

When an existing web app is published through App Manager, all the HTTP requestes to the web app go through the Gateway component of App Manager. This feature enables us to intercept the call and augment a plain web app with authentication, authorization and stat collection etc ..  

In this post, I describe how App Manager can secure an unsecured web app.

App Manager uses SAML and JWT to handle authentication scenarios.

When the gateway receives an HTTP call, it first checks whether the call is coming from an authenticated user. Gateway decides whether a requested is authenticated, by checking either the requested has a valid session associated with it, of the request has a valid SAML response from the trusted identity provider.

If the user is not authenticated he/she will be redirected to the IDP for authentication.

If the user is authenticated, a JWT will be generated, signed by the gateway and sent to the web app along with the SAML response received from the identity provider. 

JWT is a standard to securely share user claims between two parties. In our scenario, gateway is the one who handles authentication on behalf of the web app. So the web app should trust the gateway.

The configuration related to JWT in App Manager, can be found in App Manager documentation

The image below depicts the aforementioned scenario.

When the web app receives the HTTP request, it can read the JWT, which is normally sent as an HTTP header and decode it and use the user information accordingly.

e.g. The web app can extract user information and store it in the user session and use that information give a personalize view for users. 
If the web app developer has to develop the authentication part from the scratch, then it would take more time. But having App Manager in between, reduces that cost.

Decoding a JWT and verifying the signature  needs some coding. But there are a lot of libraries which do that for you. jwt.io is a pretty cool website where you can grab information about different JWT implementations in different languages.

If you are looking for a Java implementation Nimbus-JOSE-JWT is a good option.

I wrote a small Jaggery module to wrap the functionality of the above library.

Below is a code snippet of its usage.

index.jag
=======

var config = require('config.json');
var jwtClientModule = require('/modules/jwt-client.js');
var jwtClient = new jwtClientModule.JWTClient(request, config.jwtClient.headerName, config.jwtClient.certificatePath);

jwtClient.init();

log.debug("JWT = " + jwtClient.jwt);

if(jwtClient.isJWTPresent()){
  include('includes/jwt_login.jag');
}else{
  // Send an HTTP 401
}

   
jwt_login.jag
==========        

authenticateAndAuthorize();

function authenticateAndAuthorize(){

  try{
    jwtClient.parse();

    if(jwtClient.verify()){
      log.debug("Verified the signature of the JWT.");

      var subject = jwtClient.getSubject();
      var issuer = jwtClient.getIssuer();
      var claims = jwtClient.getClaims();

      setSessionAttributes(subject, claims);

      // Implement your authorization logic based on the claim values.

    }else{
      logAndShowError("Authentication failure. Cannot verify the JWT signature.");
    }
  }catch(e){
    logAndShowError("Authentication failure. Something went wrong ", e);
  }

}

function setSessionAttributes(subject, claims){

  session.put("logged-in", "true");
  session.put("username", subject);
  
  var roles = claims.get("http://wso2.org/claims/role");
  if(roles){
    roles = roles.split(',');
    session.put("roles", roles);
  }

}

Application / Subscription Sharing in WSO2 API Manager 1.9

How it works

This feature enables the users in the same organization (group) to share the applications and subscriptions. All the applications and subscriptions created by a user in an organization, are visible to the other users in the same organisation.

How does API Manager know the organization of a user ?

WSO2 API Manager Store and Publisher web apps support more than one authentication mechanism. e.g. Authentication via Carbon user stores, SAML based SSO.

As a result, the way the organization of a user should be determined accordingly.

API Manager is shipped with a default implementation for the default authentication mechanism, and API Manager has the flexibility to plug-in a different implementation. 

This article explains how application / subscription sharing can be done with the default implementation.

Enabling application / subscription sharing

Uncomment the following line in APIM_HOME/repository/conf/api-manager.xml

<GroupingExtractor>org.wso2.carbon.apimgt.impl.DefaultGroupIDExtractorImpl</GroupingExtractor>

Use case

1) A user and give his/her organization during the signing up process. ( See Fig. 1).

user_a and user_b belongs to org_1

Fig. 1

2) Then the user create a new application. (See Fig. 2)

    user_a creates app_a.

Fig. 2

3) Since user_b also belongs to org_1 he can see the application which user_a created. ( See    Fig. 3)

Fig. 3

4) user_a subscribes to WeatherAPI (See Fig. 4)

Fig. 4

5) Since user_b also belongs to org_1 he can see the subscription to WeatherAPI.

( See Fig. 5)

Fig. 5

Maven EAR plugin

Maven EAR plugin is a cool solution to assemble your EAR with modules, descriptors and libs.

Though it is bit hard to get info about the plug-in and its features, this tool can be used to assemble a fairly custom EAR as you do with an ANT script.

If the configurations are not enough for a custom assembly, you can use Maven assembly plugin which is as good as Apache ANT.

Architecture views

We kicked off a new project few weeks backs. This is very first project which I'm going to get hands-on experience in software architecture and design (in a formal way ;).

When we are wearing the developer hat, it is bit hard to accept that architecture and planning is the most important task and it cuts down a significant development time compared to a project without architecture and designin.

Building an enterprise application is complex and hard as building a large shopping complex or a warship. But surprisingly industries such as construction or electronic don't fail as much as software do !

The reason behind this fact is ARCHITECTURE.

But architecture is not a simple thing which can be achieved with minimum effort. It is collection of aspects which draw the boundaries for further design or implementation. These aspects are called Architecture Views.

There is a useful book architecture documentation from Software Engineering Institute of CMU.

Hopes this book helps you to gain a good knowledge on software architecture and formal documentation.   

Regular Expression for printable page ranges.

Last week we got a  requirement to enable printing for our student portal which was developed using Adobe Flex. And all the documents of the portal is on Flex. So our approach was to use Flex Printing.

Our print dialog box has a text box where the user can enter the range of pages to be printed (just like all other applications which has printing ;) ). I was searching for a regular expression which validates the page range pattern. While sufring web I found a cool app where you can write the regular expression and write different test cases agains that. And the interesting thing is it has been written in Flex. So if you are using Flex then you don't have to worry about platform specific stuff. (If the expression is correct in this app then it should obviously work in your Flex application.

And there is a good collection of pre-defined regular expressions which are usable in many cases. I could find one for my requirement as well. But since I needed further more restrictions I had to came up with my own one.

This is my solution :) 

/^([1-9][0-9]{0,2}|[1-9][0-9]{0,2}-[1-9][0-9]{0,2})(,([1-9][0-9]{0,2}|[1-9][0-9]{0,2}-[1-9][0-9]{0,2}))*$/

which validates a pattern like 1,2-5,6,7,12-15

You can access the aforementioned test bed using the url http://ryanswanson.com/regexp/#start