Testing The Dcsf Hub Web Service

Work Completed By: Terry Rich-Whitehead

DESCRIPTION

We have been running a project to build a new free school meals system based around the concepts of business process management and web service based integration. The project has used the singularity process platform to run each case and integrates with the Vignette records and documents system in order to create and manage the documents associated with the process.

The Department for Children, Schools and Families has issued a new draft web service for providing information on whether an individual is eligible for free school meals. This service could be potentially integrated into our new system in order to automate this particular stage of the process.

The point of this project is to test out the web service, showing how it can provide an automated response which can then be used by another application or process, documenting how this was achieved and investigating how integration into the singularity process platform can be achieved.

DETAILS/LOCATION OF PRACTICAL DEMONSTRABLE OUTPUT

Details of where code, web pages etc can be found.

Test Application

This is a small C# application that was developed to test the DCFS Hub. An install script can be found at :-

H:\ICTSharedInfo\Strategy\Apps Strategy Team\tric\DCFS Hub\setup.exe

flickr:3612846617

DCFS Hub Test App

DESCRIPTION OF THE WORK CARRIED OUT

Describe how you approached this piece of work, any technologies, tools or techniques that you found useful or tried and discarded. Include any examples that you used for inspiration and any contacts you have made in carrying out the work.

Researching Singularity

Researching Singularity proved to be a difficult task as the Singularity support pages offer no great insight into how the product functions. Fortunately an excellent overview document was found from the BPM Institute (H:\ICTSharedInfo\Strategy\Apps Strategy Team\tric\DCFS Hub\singularity.pdf). The contents of this document and talking to some WCC staff about Singularity lead me to the conclusion that it would have not been feasible to actually test the DCFS Hub from within Singularity.
However, Singularity is web enabled both as a consumer and a provider . It appears to be built around Microsoft technologies therefore it was decided that the best outcome of this project should be the development of some sample code that could be utilised by Singularity. Sample code and a small test was produced but the partly successful outcome would have never been achieved with out the use of two development aids namely soapUI and Wireshark.

SOAPUI

SOAPUI was recommended by the people at the DCFS Hub. SOAPUI exists as a free comprehensive tool for analysing, testing and simulating a web service (WSDL and REST ). Any web service that is to be tested has it’s own project which is created by opening a new project and importing the services WSDL data or importing a pre-configured soapUI project. The DCFS Hub supplied a pre-configured project which made it very straightforward to get up and running with the tool. The DCFS project plus a few additional pre-configured projects can be found at :-
H:\ICTSharedInfo\Strategy\Apps Strategy Team\tric\DCFS Hub\Sample soapUI projects
The tool itself can be download from :-
http://www.soapui.org/

Loading the project supplied by the DCFS produced the following project hierarchy :-

flickr:3612851455

The DCFS project represented a fairly typical project containing references to the ‘Main’ web service, a Test Suite and a Mock Service.

The Main section contains all the details relating to the web service, it’s endpoint, methods etc. The Main reference leads to a detailed window that provides excellent means to navigate the service.

flickr:3612851457

Service Overview

The above shows a summary of the service, it’s namespace, operation(s) etc. The most informative tab has to be the one labelled ‘WSDL Content’, selecting this provides a means to easily navigate the WSDL XML data. On the left of the form all the components are grouped under their various categories which provide an easy way to drill down in the WSDL data. Selecting any point in the WSDL hierarchy will instantly cause the related XML in the main window to be highlighted.

flickr:3612851459

Navigating the WSDL data

The Test section gives access to the tools testing features that can be used to fully test a web service. In the case of this project these tests were aimed at the service under development by the DCFS. A test is a single call to the web service. The test consists of the parameters that are to be passed to the service and the expected result. When a test is executed it will give a pass of fail. A whole series of tests can be put together as a series of Test Steps which allows the user to fully test the service. Having this feature was invaluable when problems were found developing a class in C# to access the service. At one point in the development the C# code kept failing, a quick test using soapUI revealed the cause of the problem, the service was down! Later on in the project it also helped in determining that the code generated by C# and PHP was correct.

flickr:3612851465

Running a Number of Test Steps

Another feature of the Test Suite section is the ability to carryout load tests. Running a load test provides a means to simulate use of a service in the real world, a must for developing services for use in an SOA architecture. Real time performance such as average response times etc are all displayed in a comprehensive window.

flickr:3612851469

Load Testing

The Mock Service section allows the user create a simulated service on the local machine, this has a lot of potential if the user can get it to work correctly. Unfortunately as part of this project this feature did not appear to work correctly, personally I do not think it was a fault with the product as more then likely the problem resided within the supplied WSDL data. I am drawn to this conclusion as this project revealed two problems with the WSDL data. With more time and the use of a known good service then I am sure the Mock Service can be made to work.

WireShark

WireShark is a free ‘Network Analysis’ tool that allows the developer to capture view the communication between the local PC and the remote service. This is tool proved very useful in checking XML queries generated by C# and PHP. The tool can appear a bit intimidating to start with but it does not take to long to master. WireShark gives a graphical view of every data packet entering and leaving the users machine. It not to difficult to identify the HTTP data passing in and out, one simply has to identify any one of the packets communicating with the web service and then select an option to display the whole stream. For the communication stream the user can see the call and the response.

flickr:3612834337

Following a communication stream to and from a web service

Using this tool helped in many ways. It allowed the testing of C# and PHP generated queries to be extracted and pasted in to soapUI as a test step. It also provided a means to identify other problems such as web services not responding. For example when soapUI was first used not many of the sample web services would respond. Looking at the headers it was soon found that all the HTTP headers were been sent with the version of HTTP set as 0.9 ( HTTP/0.9). Most web services were found to expect version 1.1 (HTTP/1.1). The problem in this case was soapUI as the default setting for the product was HTTP/0.9, fortunately soapUI could be configured for this version of HTTP.

Creating a C# class to access the OFSM Hub

Microsoft’s Visual Studio 2008 was used to create a class to access the DCFS web service. Creating a class to access a web service is quite straightforward one you understand how the tool works. A Web service is created by firstly adding a new ‘Reference’ to a project.

flickr:3612834341

Once the ‘Add Service Reference’ is selected a window is displayed that allows the user to explore the service. The window below depicts adding a service reference using a local WSDL file as was supplied by the DCFS Hub. Note this window only shows the methods offered by the service it does not give and indication of how they are called, parameters etc.

flickr:3612834345

Adding a Service Reference

Finding out more about the service is achieved by using the ‘Object Browser’ this reveals a lot more and gives some clues on how the services methods are used. Most web service methods are very straightforward in how they are invoked from within a section of code. The DCFS’s service involved to use of a couple of structures that are used to create the query and to handle the response. As it can be seen in the image below there are references to a ‘SubmitSingleQueryRequestStructure’ and a ‘SubmitSingleResponceStructure’ which combined give the user access to the core variables such as DateOfBirth, Surname etc.

flickr:3612834351

Using the Object Browser

Creating the call to the web service firstly involves declaring the various datatypes. the method and service itself as an object. Below is a snippet of code that does just this.

private ServiceReference1.DummyWSClient client;
private ServiceReference1.SubmitSingleQueryRequestStructure requestStruct;
private ServiceReference1.SubmitSingleQueryResponseStructure responceStruct;
private ServiceReference1.QueryData query;

All these objects are instantiated when the class it self is instantiated.


public HubService()
{
client = new Hub01.ServiceReference1.DummyWSClient();
requestStruct = new Hub01.ServiceReference1.SubmitSingleQueryRequestStructure();
responceStruct = new Hub01.ServiceReference1.SubmitSingleQueryResponseStructure();
query = new Hub01.ServiceReference1.QueryData();
}

Having declared and instantiated all the required objects it is then just a case of assigning the relevant values, making the call to the service and extracting the results. The sample code below shows all these steps, firstly assigning values to the query which is then assigned to the requestStruct. The requestStruct is then passed as a parameter to the SubmitSingleQuery method of the web service. This method returns its reply in the form of a responceStruct. The responceStruct can then be accessed to reveal an error code and eligibility status.

{{query.Surname = mSurname;
query.DateOfBirth = mDob;
query.NiNo = mNino;
query.NASS = mNass;

requestStruct.UserToQuery = query;

responceStruct = client.SubmitSingleQuery(requestStruct);

mEligibilityStatus = responceStruct.EligibilityStatus;
mErrorCode = responceStruct.ErrorCode * -1;
mErrorMessage = mErrorMessages[mErrorCode];}}

A full listing of the class can be found in the appendix. At the end of this document.

Creating a demonstration application using the OFSM Hub class

To provide a simple means to demonstrate the use of the OFSM Hub a simple Windows Desktop application was developed using Visual Studio 2008. This application is basically a form that uses the web service class to query the OFSM Hub. OFSM supplied some sample test data that can be used to test the service and this same data can be used in conjunction with this small application.

Once installed the application can be started via Start->Programs->Warwickshire County Council->Hub01.

To provide a simple means to demonstrate the use of the OFSM Hub a simple Windows Desktop application was developed using Visual Studio 2008. This application is basically a form that uses the web service class to query the OFSM Hub. OFSM supplied some sample test data that can be used to test the service and this same data can be used in conjunction with this small application.

Once installed the application can be started via Start->Programs->Warwickshire County Council->Hub01.

flickr:3612846627

OFSM Hub Test Application

Once started the following test data can be input. Simply enter data into the correct fields and select ‘Get Result’. Once selected the result of the query will be displayed, The persons eligibility for free school meals and the return code with a textual description will be displayed.

Request Data Response Data
Surname Date of Birth NiNo NASS Eligibility Status Error Code
RIG 1111-11-11 NULL <ANY> 1 0
GOO 2222-22-22 <ANY> NULL 1 0
BAD 1111-11-11 NULL <ANY> 0 0
MAL 1010-10-10 <ANY> NULL 0 0
BAD 0000-00-00 <ANY> NULL -1 -1
AAA 0000-00-00 <ANY> <ANY> -1 -2
AAA 1111-11-11 <ANY> <ANY> -1 -3
<ANY> 9999-99-99 <ANY> <ANY> -1 -4
<ANY> <ANY> A1A1A1A1A NULL -1 -5
<ANY> <ANY> NULL A1A1A1A1A1 -1 -6
<ANY> <ANY> A1A1A1A1A A1A1A1A1A1 -1 -7
ZZZ <ANY> <ANY> NULL -1 -10
If none of the above conditions are met 0 0

Sample Test Data

Below is a sample of screen shots of the application in use.

flickr:3612846617

A Successful Query

flickr:3612846619

An Error due to a invalid Date of Birth

flickr:3612846621

An Error due to an invalid National Insurance Number

PROJECT OUTCOME

Describe the degree to which the work was successful in addressing the project description. Include reasons why or why not.

SOAPIU Tool

The soapUI proved to be invaluable in this project. It is excellent for exploring a web service and the ability to create test steps would be most beneficial in an SOA development environment. I would have good to explore the use of Mock Services and their use of Groove scripts (Java). I am sure there is some more potential here that would again help in the SOA development environment.

Sample Code for Service Object

Some sample code that demonstrates the use of a web service using C#. With more time it would have been interesting if the same functionality could have been produced with PHP or Java. Attempts were made with PHP but problems were encountered in using the supplied WSDL data. The problem may be with PHP in that it expects the WSDL to be imported from the server hosting the service. Again with more time then perhaps the reason for the PHP problems could be unearthed.
There is on issue relating to this class in that no method was produced that could change the address of the remote server. I know it can be done as it been achieved in the past. The ability to change this address is fairly important as it removes the requirement to recompile the code if a remote address changes.

Hub Service Test Application

Simple desktop application developed to demonstrate the use of the web service class. It works but need a bit more testing.

SOAP Errors

A few problems were encountered with the Hub, some self inflicted others perhaps not. To make a success of this project I had to make a small modification to the information supplied. Availability was a problem at one point and there were some problems in setting up soapIU.
The problems with soapUI related to its default configuration in that it defaulted to HTTP version 0.9! At the time this did cause some confusion. Anyone using soapUI should have a look in the Appendix section.

SHORT TERM BENEFITS

What immediate impact could the output of this R&D work have on the organisation – could it provide benefits without compromising our strategic approach?

The sample class for accessing the Hub service could be used as a basses for Singularity to make use of this service.
The soapUI tool should be publicised to developers within WCC

STRATEGIC IMPLICATIONS

How the work carried out fits with our strategic direction or how it should contribute to our strategic thinking.

SOAPUI also exists in a Pro form that includes some additional features that could prove useful in developing and testing web services within WCC.

One issue that did become apparent in this project was the importance of software that is robust and bug free. In an SOA environment the quality of any software is most important and as such should be properly managed and controlled. I would expect that any services developed within WCC should also include a comprehensive test of test data that can used by automated tools every time when even the slightest change to the code is made.
Finally is the issue of response times, some web services can be a bit slow which is not very satisfactory when used in an SOA environment. Therefore I would expect that the responsiveness of a service would be an important part of any design specification and any software was developed with an attention to speed. For instance if there was a requirement for is a high usage service that only read data from a database that was fairly static then it may be worth considering holding a sub-set of this data in a memory resident database.

OTHER NOTES ETC

APPENDIX

Listing of the C# Class used to access the DCFS Hub Web Service

{{
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;

namespace Hub01
{
class HubService
{
private String mLocalAuthorityId;
private String mSurname;
private String mDob;
private String mNino = "";
private String mNass = "";
private int mEligibilityStatus;
private int mErrorCode;
private String mErrorMessage;

private string[] mErrorMessages = new string[11] {"Web service is successful","Web service failure due to an internal system error","Username and password are invalid or user not authorised","Surname is invalid","DOB is invalid","NINO is invalid format","NASS is invalid format","Both NINO and NASS supplied","8","9","-10 = Unable to complete query (DWP data unavailable)"};

private ServiceReference1.DummyWSClient client;
private ServiceReference1.SubmitSingleQueryRequestStructure requestStruct;
private ServiceReference1.SubmitSingleQueryResponseStructure responceStruct;
private ServiceReference1.QueryData query;

public HubService()
{
client = new Hub01.ServiceReference1.DummyWSClient();
requestStruct = new Hub01.ServiceReference1.SubmitSingleQueryRequestStructure();
responceStruct = new Hub01.ServiceReference1.SubmitSingleQueryResponseStructure();
query = new Hub01.ServiceReference1.QueryData();
}

private int getResultHelper()
{
query.Surname = mSurname;
query.DateOfBirth = mDob;

if (mNino.Length > 0) query.NiNo = mNino;
if (mNass.Length > 0) query.NASS = mNass;

requestStruct.UserToQuery = query;
responceStruct = client.SubmitSingleQuery(requestStruct);

mEligibilityStatus = responceStruct.EligibilityStatus;
mErrorCode = responceStruct.ErrorCode * -1;

mErrorMessage = mErrorMessages[mErrorCode];
return mErrorCode;
}

public int getResult()
{
return this.getResultHelper();
}

public int getResult(String Surname,String Dob)
{
mSurname = Surname;
mDob = Dob;

return getResultHelper();
}

public String ErrorMessage
{
get
{
return mErrorMessage;
}
}

public int EligilityStatus
{
get
{
return mEligibilityStatus;
}
}

public String Nass
{
set
{
mNass = value;
}
}

public String Nino
{
set
{
mNino = value;
}
}

public String Dob
{
set
{
mDob = value;
}
}

public String Surrname
{
set
{
mSurname = value;
}
}
}
}
}}

SoapIU
A good product but watch out for some gotcha’s
To start soapUI navigate to its installed directory then start the bat file soapui.bat from the bin directory.

Proxy Settings

External web services cannot be accessed without setting the Proxy Settings. Navigate to the File->Preferences this will launch the soapUI Preferences window. Use the ‘Proxy Settings’ tab and set the proxty settings.

flickr:3612846625

Incorrect HTTP transport protocol version

SoapIU’s default HTTP transport version is 0.9! a lot of web services do not handle this version. SoapUI will throw an exception but it does not give must of a clue to what the source of the problem was as the results screen will not show any data. Only be use of a network analyser exposed the message “<h1>Length Required</h1>” that was actually returned. Not that gives much away but it does show that the server responded. So far when this message has been returned then it is an indication the http transport protocol version was incorrect..
Fixing this problem is simple. Navigate to the File->Preferences this will launch the soapUI Preferences window. Use the HTTP Setting tab and change the ‘HTTP Version’. Unfortunately changing these settings effects ALL the users projects!

flickr:3612846623

Setting the HTTP transport version

Unless otherwise stated, the content of this page is licensed under Creative Commons Attribution-ShareAlike 3.0 License