Archive for the 'Development' Category

Using HP TRIM Web Service to download files

Published on December 20, 2011 in Development. Closed

So recently I’ve had to download files from HP TRIM using the web service. Replace the record number with the record number you’re searching for. The endpoint should be:

http://serverwithtrimwebserviceinstalled/dataset/trim.asmx, where dataset is the name of your dataset.

There’s not much information available on how to do this, so here’s my message straight from SoapUI:

<soap:Envelope xmlns:soap=”http://www.w3.org/2003/05/soap-envelope” xmlns:trim=”http://www.towersoft.com/schema/webservice/trim2/”>
<soap:Header/>
<soap:Body>
<trim:Execute>
<!–Optional:–>
<trim:req>
<trim:ShortcutRecordNumber>
<!–Optional:–>
<trim:TargetObjectType>?</trim:TargetObjectType>
<trim:TargetForUpdate>false</trim:TargetForUpdate>
<!–Optional:–>
<trim:Id>?</trim:Id>
<trim:IsForUpdate>false</trim:IsForUpdate>
<trim:Limit>1</trim:Limit>
<!–Optional:–>
<trim:RecordNumber>D-41923</trim:RecordNumber>
</trim:ShortcutRecordNumber>
<trim:Download>
<!–Optional:–>
<trim:TargetObjectType>record</trim:TargetObjectType>
<trim:TargetForUpdate>false</trim:TargetForUpdate>
<!–Optional:–>
<trim:Id/>
<trim:Checkout>false</trim:Checkout>
<!–Optional:–>
<trim:Comments>?</trim:Comments>
<!–Optional:–>
<trim:CheckoutPath>?</trim:CheckoutPath>
<trim:MaximumTransferBytes>0</trim:MaximumTransferBytes>
<trim:TransferInset>0</trim:TransferInset>
<!–Optional:–>
<trim:ChildUri>?</trim:ChildUri>
<trim:TransferType>inline</trim:TransferType>
<!–Optional:–>
<trim:DownloadId/>
</trim:Download>
<trim:HideVersionNumbers>1</trim:HideVersionNumbers>
<trim:ProvideTimingResults>1</trim:ProvideTimingResults>
<trim:ForceRealTimeCacheUpdate>1</trim:ForceRealTimeCacheUpdate>
</trim:req>
</trim:Execute>
</soap:Body>
</soap:Envelope>

 

Which will return:

<soap:Envelope xmlns:soap=”http://www.w3.org/2003/05/soap-envelope” xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xmlns:xsd=”http://www.w3.org/2001/XMLSchema” xmlns:wsa=”http://schemas.xmlsoap.org/ws/2004/08/addressing” xmlns:wsse=”http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd” xmlns:wsu=”http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd”>
<env:Header xmlns:env=”http://www.w3.org/2003/05/soap-envelope”>
<wsa:Action>http://www.towersoft.com/schema/webservice/trim2/ExecuteResponse</wsa:Action>
<wsa:MessageID>urn:uuid:9d33d037-a2eb-4ce8-8feb-6b92146724a6</wsa:MessageID>
<wsa:RelatesTo>urn:uuid:bb12ff33-35cd-4bef-9a2f-11eebc7b93bd</wsa:RelatesTo>
<wsa:To>http://schemas.xmlsoap.org/ws/2004/08/addressing/role/anonymous</wsa:To>
<wsse:Security>
<wsu:Timestamp wsu:Id=”Timestamp-c6e82623-d2ee-443f-8615-a8ba211ab2ac”>
<wsu:Created>2011-12-19T20:48:24Z</wsu:Created>
<wsu:Expires>2011-12-19T20:53:24Z</wsu:Expires>
</wsu:Timestamp>
</wsse:Security>
</env:Header>
<soap:Body>
<ExecuteResponse xmlns=”http://www.towersoft.com/schema/webservice/trim2/”>
<ExecuteResult>
<TimingResult>
<Id>?</Id>
<TimeConsumed>18750120</TimeConsumed>
</TimingResult>
<SearchResult>
<Id>?</Id>
<FoundCount>1</FoundCount>
</SearchResult>
<TimingResult>
<Id/>
<TimeConsumed>625004</TimeConsumed>
</TimingResult>
<DownloadResult>
<Id/>
<Base64Payload>(document inline)</Base64Payload>
<DateTimeModified>11/11/2011 14:16:27</DateTimeModified>
<DocumentSize>28901</DocumentSize>
<FileType>DOCX</FileType>
<IsTrimMail>false</IsTrimMail>
<TransferType>inline</TransferType>
<AttachmentId>record-5332</AttachmentId>
<FinalChunk>false</FinalChunk>
</DownloadResult>
<EndResponse/>
</ExecuteResult>
</ExecuteResponse>
</soap:Body>
</soap:Envelope>

Opa Web Chat Tutorial – Displaying the user name to the user

Published on October 26, 2011 in Development. 1 Comment Tags: , .

So I’ve been doing the Opa web chat tutorial @ http://doc.opalang.org/ and one of the tasks for you to achieve is to customize the chat so that at startup at the start of the div named #conversation, display the message “Hello, you are user xxx” where xxx is the random author name.

This was quite a challenge for me until I read the part about what do means. Basically…

Functions can take 0 or more arguments, and return one and only one value. Think of it like an automatic return statement on the first calcuation that isn’t defining a value, or has a do in front of it.

So in the end inside the start() function, I have the following line:

<div id=#conversation onready={
_  -> (
do Dom.put_at_start(#conversation, Dom.of_xhtml(<div>Hello you are user {author}</div>))
Network.add_callback(user_update, room)
)
}> </div>

Which states when the #conversation div is ready to start processing, execute the statement in the curly brackets. We then start the execution of a function that has no name (we don’t care what the name is, it’s a throw-away language construct), and in the curved brackets we describe what the function should do.

Going back to the bit about do’s at the beginning of the article, this function has two statements:

Dom.put_at_start(#conversation, Dom.of_xhtml(<div>Hello you are user {author}</div>))
Network.add_callback(user_update, room)

But due to Opa returning the first value it can in a function, imagine that the function really looks like in say php:

function() {
return Dom.put_at_start(#conversation, Dom.of_xhtml(<div>Hello you are user {author}</div>));
return Network.add_callback(user_update, room);
}

Which of course makes no sense. So you need to put a do in front of one of those statements to nullify the return that’s automatically put there, making our function look like:

function() {
Dom.put_at_start(#conversation, Dom.of_xhtml(<div>Hello you are user {author}</div>));
return Network.add_callback(user_update, room);
}

Hooray!

A surefire way to let business drive IT projects

Published on March 2, 2011 in Business, Career Development, Development, E-Commerce and Great Ideas. Closed Tags: , .

I’ve been thinking a bit about good methods to help the business (referred to as anyone that’s not IT) drive IT projects.

So far the method I’ve observed goes along the lines of:

1. Define the problem. Sounds like an obvious step, but in IT we’re all too keen to jump to solutions. A problem isn’t that we need an ERP or a CRM system, any more than not having a hamburger is a problem. The real problem is you’re hungry. Or for the first two examples, you want to manage the resources of an organisation efficiently, or you want to track all your interactions with your customers across multiple communication channels and plan marketing campaigns.

With the problem defined we can move on to…

2. Goals. Goals are visionary statements that frame any potential solution. Think of them as solution boundaries, where the solution must sit inside. A solution scope I guess. One could be more concise information for managers. These goals are intangible and immeasurable in the sense that it’s difficult to know if you’ve achieved them, but without them your solution might not fit into the strategic direction of the enterprise. No point coming up with a mobile solution if there’s no need for it.

3. Business objectives. These are finite measurable objectives that sum up how we know the problem has been solved in concrete terms. 100% of senior management reports come from the data warehouse. Customers will be able to purchase items from mobile devices. All absolutely measurable. No ambiguity, not for the business who can say that these objectives if met will meet the current needs of the business, nor for IT and the vendors who have no wriggle room in terms of what they’re delivering at the other side. As you can see, scope at this point if already defined, and while things could change within the boundary, if work doesn’t directly support the solving of the objective then it shouldn’t be in the project.

4. Functional use cases. The creation of these further tighten your scope and clarify exactly how people, process, and technology will change to meet the business objectives. Simple use case diagrams are great for communicating with the business the functions and processes within the solution, and we haven’t even talked any technology yet. After all, the solution could be solved by just a process change. Once all the functional use cases required to meet the business objective have been agreed to by the business, the scope is effectively defined. Then these use cases in the diagram can each be turned into fully dressed use cases with the standard process, exceptions, inputs and outputs, and all information a good use case should have. An example could be Load Information or Create Campaign.

5. Business requirements. With our functionality defined and processes sorted, we can think about what technological requirements are required to achieve the use cases. Creating a campaign will require software that has the ability to create a campaign. But we’re not focusing on a specific technology, and so the business requirements must be technology agnostic, and not just written as a laundry list of features of a particular software suite you have in mind. Because that would still be jumping to a solution ahead of time.

6. Solutions architecture. This really defines the non functional requirements of the solution, and is likely to start formulating a solution to meet the objectives. This would include aspects of solution analysis and design, and takes in the input of business requirements and comes up with a solution design. Sure some of the implementation gaps may not be filed, but by this point in time a solution had now been defined and can be traced all the way back to the problem.

From here there’s the normal software implementation life cycle, including change and release management procedures. By following this process we can ensure that we’re solving business problems in a clearly defined traceable manner that the business can understand and accept.

Eurekastreams Ubuntu 10.04 LTS Installation Guide

Published on August 12, 2010 in Development. 2 Comments Tags: , .

So you’ve heard about Eureka Streams, the open source social networking platform from Lockheed Martin @ http://www.eurekastreams.org and you’d like to check it out. Here’s how.

  1. Start with a virtual machine (http://www.virtualbox.org/wiki/Downloads to download VirtualBox);
  2. Download the Ubuntu 10.04 LTS ISO (http://www.ubuntu.com/desktop/get-ubuntu/download) and set it up according the defaults. I did give it 100GB of disk space, expanding dynamically;
  3. At the command line type ‘sudo apt-get install nano‘ to install a command line text editor;
  4. At the command line type ‘sudo nano /etc/apt/sources.list‘ and paste in the text from http://kenning.co.nz/wp-content/uploads/2010/08/apt-get-sources.txt which will make sure you can pick packages from the universe repositories;
  5. At the command line type ‘sudo apt-get update‘;
  6. At the command line type ‘sudo apt-get upgrade‘;
  7. At the command line type ‘sudo apt-get install memcached‘;
  8. At the command line type ‘sudo apt-get install default-jdk‘;
  9. At the command line type ‘sudo apt-get install git-core‘;
  10. At the command line type ‘sudo apt-get install postgresql‘;
  11. At the command line type ‘sudo apt-get install maven2‘;
  12. Edit /etc/environment and add ‘JAVA_HOME=/usr/lib/jvm/default-java‘;
  13. Log out and log back in;
  14. At the command line type ‘echo $JAVA_HOME‘ and make sure the above
    location is returned;
  15. At the command line type ‘git clone https://git@github.com/lmco/eurekastreams.git‘;
  16. At the command line, change to the eurekastreams folder created;
  17. At the command line type ‘mvn install:install-file -Dfile=lib/shindig-common-3.3.jar -DgroupId=org.apache.shindig -DartifactId=shindig-common -Dversion=3.3 -Dpackaging=jar -DgeneratePom=true‘;
  18. At the command line type ‘mvn install:install-file -Dfile=lib/shindig-features-3.3.jar -DgroupId=org.apache.shindig -DartifactId=shindig-features -Dversion=3.3 -Dpackaging=jar -DgeneratePom=true‘;
  19. At the command line type ‘mvn install:install-file -Dfile=lib/shindig-gadgets-3.3.jar -DgroupId=org.apache.shindig -DartifactId=shindig-gadgets -Dversion=3.3 -Dpackaging=jar -DgeneratePom=true‘;
  20. At the command line type ‘mvn install:install-file -Dfile=lib/shindig-server-3.3.war -DgroupId=org.apache.shindig -DartifactId=shindig-server -Dversion=3.3 -Dpackaging=war -DgeneratePom=true‘;
  21. At the command line type ‘mvn install:install-file -Dfile=lib/shindig-social-api-3.3.jar -DgroupId=org.apache.shindig -DartifactId=shindig-social-api -Dversion=3.3 -Dpackaging=jar -DgeneratePom=true‘;
  22. At the command line type ‘mvn install:install-file -Dfile=lib/shindig-common-3.3-tests.jar -DgroupId=org.apache.shindig -DartifactId=shindig-common -Dversion=3.3 -Dpackaging=test-jar -DgeneratePom=true‘;
  23. At the command line type ‘mvn install:install-file -Dfile=lib/gwt-dnd-2.6.5.2.jar -DgroupId=com.google.code.gwt-dnd -DartifactId=gwt-dnd -Dversion=2.6.5.2 -Dpackaging=jar -DgeneratePom=true‘;
  24. At the command line type ‘mvn install:install-file -Dfile=lib/carousel-0.6.1.jar -DgroupId=com.gwtyuicarousel -DartifactId=carousel -Dversion=0.6.1 -Dpackaging=jar -DgeneratePom=true‘;
  25. At the command line type ‘mvn install:install-file -Dfile=lib/urlrewrite-3.2.0-beta.jar -DgroupId=org.tuckey -DartifactId=urlrewrite -Dversion=3.2.0-beta -Dpackaging=jar -DgeneratePom=true‘;
  26. At the command line type ‘mvn install‘;
  27. At the command line type ‘mvn clean install -Dopenaccess-security -Dbuild.web.baseurl=http://localhost:8080 -Dbuild.web.grbaseurl=http://localhost:8081‘;
  28. Change to the ‘web/target/classes/db/scripts‘ folder;
  29. At the command line type ‘sudo -u postgres psql postgres‘;
  30. At the PSQL command line type ‘\password postgres‘ and pick a password for the postgres account;
  31. At the command line type ‘sudo -u postgres createdb eurekastreams‘ to create a eurekastreams DB;
  32. At the command line type ‘sudo pico dev_reset_db.sh‘ and search for ‘scripts$‘ and then remove the trailing dollar sign;
  33. At the command line type ‘sudo pico dev_apply_updates.sh‘ and search for ‘scripts$‘ and then remove the trailing dollar sign;
  34. At the command line type ‘sudo nano /root/eurekastreams/web/target/classes/db/schema/eurekastreams_1_databasecreate.sql‘ (or where ever you installed eurekastreams) and search for the word ‘template1′ and replace with ‘template0′. This will stop the database creation step failing because of encoding differences;
  35. At the command line type ‘sudo -u postgres createuser –superuser $USER‘ to create a postgresql account as the user you’re using because you’ll need to be a postgresql super user to be able to run the dev_reset_db.sh script – note there’s two dashes before superuser;
  36. At the command line type ‘sudo -u postgres psql‘;
  37. At the PSQL command line type ‘\password $USER‘ where $USER is the user you’ve logged in as and give your postgresql user a password;
  38. At the command line type ‘./dev_reset_db.sh eurekastreams eurekastreams localhost‘, make sure you have a eurekastreams DB but no eurekastreams SQL account;
  39. The command will execute. It will drop the existing eurekastreams DB, and then create a eurekastreams SQL account. It may then ask you for a password for the eurekastreams account;
  40. If it does, at the command line type ‘sudo apt-get install pgadmin3‘;
  41. At the command line type ‘pgadmin3‘;
  42. In PGAdmin, log in as the ‘postgresql’ account with the password you entered at step 30;
  43. Right click on the eurekastreams user, and reset the password to something you know;
  44. Then go back to the dev_reset_db script and enter the password you’ve created for the script. The script won’t accept a blank password;
  45. The script should then populate the eurekastreams DB;
  46. At the command line type ‘sudo pico /etc/postgresql/8.4/main/pg_hba.conf‘;
  47. Change the ‘local all all ident‘ line to ‘local all all trust‘ line and save the file;
  48. At the command line type ‘sudo /etc/init.d/postgresql-8.4 restart‘ to restart the SQL server;
  49. Change back to the eurekastreams/web folder;
  50. At the command line type ‘mvn jetty:run -DskipTests -Dlog.home=$HOME/logs -Denv.WEBPORT=8080 -Denv.GRWEBPORT=8081‘ and wait for the Jetty server to start;
  51. Head to http://localhost:8080 and type the username ‘sysadmin’ and a blank password.

It should now log you in!

Is JMeter URL encoding everything through the HTTP Proxy? This is how to fix it!

Published on May 21, 2010 in Development. Closed Tags: , , .

I was having problems with JMeter URL encoding everything through the HTTP proxy which was causing me serious grief. One application I’m performance testing sends XML over HTTP, which when URL encoded turns <” symbols into %3Es and other weird things that make servers panic and fail.

I made sure that encoding was switched off when the sampler was run, but that didn’t fix the problem. I checked data flowing through the JMeter HTTP Proxy and encoding was happening there compared to the normal application. Wireshark was great to figure this out.

The problem? The sampler HTTP Request HTTPClient. This is the Apache sampler and is generally better than HTTP Request in most situations I’ve been told. This is one situation where it wasn’t. I switched to HTTP Request, and all my dramas went away.

Extracting variables using Regular Expressions in JMeter

Published on May 5, 2010 in Development. Closed Tags: , , .

I’m trying to web performance test an application that regularly changes URLs. We’ll use regular expressions to figure out what that URL should be, store it as a JMeter variable, and use it in a sampler.

So first of all, we do a search for a book at a known page at the following address /cgi-bin/.exe/ENQ/CAT/BIBENQ. This address and the screenshots have been edited to remove the application name and the host address. From the result of this we’ve received some data as you can see in the second screenshot.

This data comes in the format of a URL: /cgi-bin/.exe/FULL/CAT/BIBENQ/XXXXX/410986,1, where XXXXX is a five digit number that changes constantly. This means we can’t set up a sampler with a static address, as the address won’t always be correct.

So for the sampler with the result data, we’ll right click and add Post Processor > Regular Expression Extractor. The fields in details are:

  • Reference Name – This is the variable name, in my case FLUTEURL. Elsewhere in JMeter, we’ll reference this variabvle as ${FLUTEURL}
  • Regular Expression – This is the regular expression that will extract the data we’re looking for. Brackets are put around the value we want to extract. So in our example we have /cgi-bin/.exe/FULL/CAT/BIBENQ/(.+?)/410986,1 or, we’re looking for the value that sits between the slash after BIBENQ and the slash before 410986,1. This will return in our case a five digit number, XXXXX
  • Template – Your regular expression could have multiple values extracted, in our case we have just one, and so we’re using the first template, or $1$
  • Match No. – If there are multiple matches, which one should be extracted? In our case, we want the first match, or 1
  • Default value – If a value cannot be extracted, what should we put in place instead? You could have ERROR, but I just leave it blank

So we now have a JMeter variable called ${FLUTEURL} with a value of say, 12345. How would we use this? In my case, for the path for the next sampler is:

  • /cgi-bin/.exe/FULL/CAT/BIBENQ/${FLUTEURL}/410986,1

So in execution the URL the sampler requests is /cgi-bin/.exe/FULL/CAT/BIBENQ/12345/410986,1 and the sampler works correctly.

In summary, we’ve extracted a value from a page, stored it as a JMeter variable, and then used that variable in another sampler.

Using the HTTP Proxy in JMeter to record web interactions

Published on May 5, 2010 in Development. Closed Tags: , .

I’ve been trying to figure out the performance of an application that runs over HTTP. Sounds like a perfect candidate for JMeter!

In the past, I’ve had to deal with native HTTPS applications which means I couldn’t use the HTTP Proxy to record sessions. However, since this application functions in both HTTP and HTTPS modes, this was my chance to understand and use the HTTP Proxy, so let’s go through how to set up and do this.

First, right click on the WorkBench and add the Non Test Element HTTP Proxy Server. Non Test Elements only appear when you right click on the WorkBench, so if you’ve never seen them before, now you know why.

Here are the options I recommend:

  • Port – I use 8000 just because I might be running an Application Server on port 8080. Any port over 1024 will see you right.
  • Target Controller – Select Use Recording Controller. We’ll add a Recording Controller just after this.
  • Grouping – I use Store 1st sampler of each group only just because I don’t want to save javascript or image files, just the HTML file being served. You may wish to pick a different option if you want to know the total amount of time to transfer all files to the end user for your performance test.
  • Capture HTTP Headers – I ticked it.
  • Add Assertions – I ticked it. Adds a blank assertion for each sampler. Since you’ll be using assertions later on to determine how requests are going, save time by adding a blank one now.
  • HTTP Sampler Type: HTTP Request HTTPClient. This is using the Apache HTTP Request library over the standard Java library.
  • Follow Redirects – I ticked it. This depends on the behaviour of your web application.
  • Redirect Automatically – I left blank. This also depends on the behaviour of your web application.

There are content filters you can add to stop sampling specific file types such as GIF and JPG, but since I just store the first sampler anyway, I normally leave these blank. Same with URL Pattersm to Exclude.

Right click on WorkBench and add Logic Controller > Recording Controller. All samplers recorded will be stored under here.

My useful tip – right click on the HTTP Proxy Server and add Listener > View Results Tree. This way you can analyse the requests being recorded by the HTTP Proxy.

Finally go back to the HTTP Proxy and click Start. All requests going through the HTTP Proxy on port 8000 are now being recorded.

Now head to your favourite web browser and set the HTTP proxy settings for localhost on port 8000. Anything you do now in that browser will be recorded by JMeter.

As you can see in the View Results Tree listener, you can see all the requests being made by the browser, such as:

  • the Sampler name
  • the Sampler result
  • the HTTP response code
  • Request information showing the actual request, and headers associated
  • And the actual response data, rendered as HTML

When you expand the Recording Controller, you’ll see the HTTP Request with the:

  • Server name
  • Protocol (HTTP, HTTPS)
  • Method (GET or POST)
  • Parameters (variables sent in the request)

All filled in for you. You’ll even see a HTTP Header Manager all filled in for you as well. Some web applications pay close attention to this, so it’s good to see all the information a normal browser sends.

So in summary, we’ve set up an HTTP Proxy to record sessions going through JMeter, and we’ve highlighted some of the options and what they do.

Exchange 2007 Load Generator tool hanging when installing – a fix

Published on July 17, 2009 in Development. Closed Tags: .

If you’re having problems with Exchange 2007 Load Generator tool hanging when installing, try disabling the network card during the installation. It’s weird but it works!

The problem – being able to interact with your hands with a 3D hologram

Published on June 20, 2009 in Development, Great Ideas and Usability. Closed

The issues

When you put your hand into a hologram, and grab something and move it away, how does the computer know what you’re aiming at?

I guess this is a problem of 3D space, and knowing exactly

- what is being displayed

- where each pixel of light is

- a 3d model of the hand

For example, if someone is touching a cube that is displayed in midair, we need to know all the points of the cube, and where they exist in the 3D space of the real world. So we imagine that the cube is a physical cube. Once we know it exists, we need to model other 3D objects that exist in that 3D world, such as a hand. Once we have a hand modelled and mapped to the 3D world, we can then model these two 3D objects interacting with each other, even though one isn’t real (this doesn’t matter).

So now that the computer knows where two objects are, that doesn’t mean to say a person knows where his hand is in relation to an object. This depends on their perspective, i.e. where their head is in relation to their eyes, in relation to the object they’re interacting with.

The next thing is to make sure we’re interacting with the object we think we’re interacting with. For instance, we could be touching one box, but it’s really in the position of the second box. This is because of the angle that we’re viewing this from. So we need to look at the person that is interacting with an object, and trace a line from their sight of vision to their hand to the object they are touching.

Resourcer graphs/screenshots

Published on May 12, 2009 in Development and Usability. Closed Tags: .

I’m working on how to present data entered into Resourcer.I’ve been playing around with Excel and thinking about how to represent:

  • Tasks done;
  • Their utilisation level;
  • Time spent on each task.

Here’s a mock up of a typical day at work:

graph

At a glance you can see that tasks with low utilisation levels are either relatively small or relatively large, while tasks where I am busy or very busy are around the 60 minute mark.