Getting XenServer VM metrics in Java

XenServer is a product of Citrix that can run virtual machines on a set of physical hosts called a pool. There is an API that allows you to create programs that interact with the XenServer pool. There are some function calls that return metrics of the virtual machines and the physical hosts.

Let us have a look at retrieving the CPU load of a virtual machine (VM). This can be accomplished with the following API call (assuming that vm is of type com.xensource.xenapi.VM and connection is of type com.xensource.xenapi.Connection):

vm.getMetrics(connection).getVCPUsUtilisation(connection)

However, as many have noted, the answer is always 0.0. As of XenServer version 5.5, the way these metrics are stored has changed. The preferred way to get VM metrics now, is to make a direct HTTP call to the physical hosts. This call will return RRD XML files. This makes things more difficult because now there are multiple calls to make, one to the XenServer API and one call per physical host machine. And one needs to dig into these files to get the CPU load, instead of having the value returned in a single call.

There is a way to make the XenServer API behave like it did in version 5.0 and prior. We start by logging into the console of a physical machine and entering:

xe host-list

Note the UUID values of all the physical host machines. Now enter the following command for each UUID:

xe host-param-set uuid=<uuid found in previous command> other-config:rrd_update_interval=1

Now reboot all physical host machines and the VM metric command will work like a charm.

Posted in Cloud computing, programming | Tagged , | 3 Comments

Using OpenJMS within Eclipse

OpenJMS allows you to use the Java Message Service (JMS) without a full-fledged Java Enterprise Edition container. It is an open source implementation of the JMS 1.1 specification.

We start by installing and starting OpenJMS. On Windows this is done by extracting the .zip file, going to the bin directory and calling startup.bat. On Linux this is done by extracting the .tar.gz file, going to the bin directory and calling startup.sh.

The next part is creating a Java project in Eclipse. Make sure to create the following directories:

  • bin, containing your own Java .class files
  • etc, containing the JNDI configuration file
  • lib, containing all required .jar library files
  • src, containing your own Java .java files

Start by copying all .jar files from the OpenJMS distribution to the lib directory and tell Eclipse to use these .jar files. Also add the etc directory to the build path. This ensures that the JNDI configuration file that we will create shortly is used at runtime.

Now create the JNDI configuration file that will tell our application where to find the running OpenJMS server. Create a file called jndi.properties with the following content:

java.naming.factory.initial=org.exolab.jms.jndi.InitialContextFactory
java.naming.provider.url=tcp://localhost:3035

Finally, we move on to the actual Java code that makes use of this groundwork. We create a message sender that sends a message to the topic topic1. This is a topic name that is available in OpenJMS by default.

MessageSender.java

import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.Destination;
import javax.jms.MessageProducer;
import javax.jms.Session;
import javax.naming.Context;
import javax.naming.InitialContext;

public class JMSSender {

   public void start() {
      Context context = null;
      Connection connection = null;
      MessageProducer sender = null;
      try {
         context = new InitialContext();
         ConnectionFactory factory = (ConnectionFactory) context.lookup("ConnectionFactory");
         connection = factory.createConnection();
         connection.start();

         Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
         Destination destination = (Destination) context.lookup("topic1");
         sender = session.createProducer(destination);
         sender.send(session.createTextMessage("Hello World!"));
      } catch (Exception e) {
      } finally {
         try {
            if (sender != null) sender.close();
            if (connection != null) connection.close();
            if (context != null) context.close();
         } catch (Exception e) {
         }
      }
   }

   public static void main(String[] args) {
      JMSSender jmsSender = new JMSSender();
      jmsSender.start();
   }
}

MessageReceiver.java

import javax.jms.Connection;
import javax.jms.ConnectionFactory;
import javax.jms.Destination;
import javax.jms.JMSException;
import javax.jms.Message;
import javax.jms.MessageConsumer;
import javax.jms.MessageListener;
import javax.jms.Session;
import javax.jms.TextMessage;
import javax.naming.Context;
import javax.naming.InitialContext;
import javax.naming.NamingException;

public class JMSReceiver {

   public void start() throws JMSException, NamingException {
      Context context = null;
      Connection connection = null;
      MessageConsumer receiver = null;
      try {
         context = new InitialContext();
         ConnectionFactory factory = (ConnectionFactory) context.lookup("ConnectionFactory");
         connection = factory.createConnection();
         Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
         Destination destination = (Destination) context.lookup("topic1");

         receiver = session.createConsumer(destination);
         receiver.setMessageListener(new MessageListener() {
            public void onMessage(Message message) {
               TextMessage text = (TextMessage) message;
               try {
                  System.out.println("Received message: " + text.getText());
               } catch (JMSException e) {
               }
            }
         });

         connection.start();
      } catch (Exception e) {
      }
   }

   public static void main(String[] args) {
      JMSReceiver jmsReceiver = new JMSReceiver();
      jmsReceiver.start();
   }
}

Now start the JMSReceiver application and then the JMSSender application. If all goes well, the JMSReceiver application should print the text:

Received message: Hello World!
Posted in programming | Tagged , , , | Leave a comment

Using the sage mathematical software system


This article describes the installation and simple usage of the Sage mathematical software system. Sage is free, open-source math software that supports research and teaching in algebra, geometry, number theory, cryptography, numerical computation, and related areas.

We start by installing Sage on Ubuntu:

sudo apt-get install sagemath

Let’s take a look at what the Sage commandprompt can give us:

sage

This will present you with a prompt that says “sage:”. You can type things here that Sage will try to understand and solve. For example, if we type “2+3”, Sage will respond with “5”:

sage: 2+3
5

Now let’s try to solve something a bit more difficult:

sage: a, b = var('a, b')
sage: solve([a + b == 12, a - b == 6], [a, b])
[[a == 9, b == 3]]

There are numerous functions you can use. We will show some of them here:

sage: factor(60)
2^2 * 3 * 5

sage: is_prime(1234567891)
True

sage: gcd(100, 24)
4

Using some Python magic, we can do things like:

sage: [factor(n) for n in range(2, 10)]
[2, 3, 2^2, 5, 2 * 3, 7, 2^3, 3^2]
Posted in programming | Tagged | 1 Comment

Examples of public SOAP web services

soap

In the previous article we showed how to use the suds library in Python to access SOAP web services. Here are some extra examples of public SOAP web services. Some of the outputs are edited for readability. We start with a recap of the one we used in the previous article.

Map IP address to country

import suds

url = "http://www.ecubicle.net/iptocountry.asmx?wsdl"
client = suds.client.Client(url)

print client.service.FindCountryAsString("194.145.200.104")
<?xml version="1.0"?>
<IPAddressService>
   <country>Netherlands</country>
</IPAddressService>

Country Information

import suds

url = "http://webservices.oorsprong.org/websamples.countryinfo/CountryInfoService.wso?WSDL"
client = suds.client.Client(url)

print client.service.ListOfCountryNamesByCode()
print client.service.CountryISOCode("Netherlands")
print client.service.CapitalCity("NL")
print client.service.CountryCurrency("NL")
(ArrayOftCountryCodeAndName){
   tCountryCodeAndName[] =
      (tCountryCodeAndName){
         sISOCode = "AD"
         sName = "Andorra"
      },
...
     (tCountryCodeAndName){
         sISOCode = "ZW"
         sName = "Zimbabwe"
      },
 }

NL

Amsterdam

(tCurrency){
   sISOCode = "EUR"
   sName = "Euro"
 }

Whois Service

import suds

url = "http://www.ecubicle.net/whois_service.asmx?WSDL"
client = suds.client.Client(url)

print client.service.Whois("whois.tucows.com", 43, "google.com")
Whois Server Version 2.0

Domain names in the .com and .net domains can now be registered
with many different competing registrars. Go to http://www.internic.net
for detailed information.

   Domain Name: GOOGLE.COM
   Registrar: MARKMONITOR INC.

...

ISBN Test

import suds

url = "http://webservices.daehosting.com/services/isbnservice.wso?WSDL"
client = suds.client.Client(url)

print client.service.IsValidISBN13("9789059650886")
True

US Zip Validator

import suds

url = "http://www.webservicemart.com/uszip.asmx?WSDL"
client = suds.client.Client(url)

print client.service.ValidateZip("90210")
<result code="200">
   <item
      zip="90210"
      state="CA"
      latitude="34.0888"
      longitude="-118.40612"
   />
</result>

Number Conversion

import suds

url = "http://www.dataaccess.com/webservicesserver/numberconversion.wso?WSDL"
client = suds.client.Client(url)

print client.service.NumberToWords(2931)
two thousand nine hundred and thirty one
Posted in programming | Tagged , , | Leave a comment

Python SOAP client with suds

suds
The library suds allows Python to make SOAP calls (that is, Python is the web service client).

We start by installing the suds library on an Ubuntu machine. The Python setuptools are needed to install suds.

sudo apt-get install python-setuptools

Then we download, unpack and install suds.

wget https://fedorahosted.org/releases/s/u/suds/python-suds-0.3.7.tar.gz
tar -zxvf python-suds-0.3.7.tar.gz
cd python-suds-0.3.7
sudo python setup.py install

The library is now ready to use. We start by importing the suds library, creating a client based on a SOAP url, and asking the library to print the SOAP web service methods that are available to us.

import suds
url = "http://www.ecubicle.net/iptocountry.asmx?wsdl"
client = suds.client.Client(url)
print client

From the output of the last print command, we learn that there is a method called FindCountryAsString that takes one argument: the IP address.

print client.service.FindCountryAsString("194.145.200.104")

And it shows (edited for readability):

<?xml version="1.0"?>
<IPAddressService>
   <country>Netherlands</country>
</IPAddressService>

Normally you want to have the contents of the SOAP body. This is what suds provides in a very elegant way. However, you’re a bit stuck when you want to get something from the SOAP header. The author of suds realised this and made a backdoor to get the information anyway. We start by showing what the function last_received contains:

print client.last_received()
<?xml version="1.0" encoding="UTF-8"?>
<soap:Envelope>
   <soap:Header>
      <ResponseHeader xmlns="">
         <resultCode>1000</resultCode>
         <resultDescription>Success</resultDescription>
      </ResponseHeader>
   </soap:Header>
   <soap:Body>
...
   </soap:Body>
</soap:Envelope>

We can get portions of this data by doing some XML handling. Let’s say we want to print the resultCode:

print client.last_received().getChild("soap:Envelope").getChild("soap:Header").getChild("ResponseHeader").getChild("resultCode").getText()
Posted in programming | Tagged , , | 13 Comments

Using Python to add new posts in WordPress

wordpress
Wordpress is a web publishing platform, which allows you to add and edit content to your website. This can be done by visiting the administration interface, wp-admin. However, there is also an XML RPC programming interface that allows you to do the same from any programming language that supports this interface.

In this short article I will show how to add new posts to WordPress from Python. There are several libraries available that allow you to view and edit content, but they were all inappropriate for me. I wanted to be able to convert my old website into WordPress posts and have the published date set to the date I originally posted items on my old site. However, every library I tried either didn’t have the possibility to set the published date, or WordPress didn’t understand it.

Here’s the short piece of code that allowed me to add new posts to WordPress, without the use of any 3rd party library:

import datetime, xmlrpclib

wp_url = "http://www.example.com/xmlrpc.php"
wp_username = "someuser"
wp_password = "secret"
wp_blogid = ""

status_draft = 0
status_published = 1

server = xmlrpclib.ServerProxy(wp_url)

title = "Title with spaces"
content = "Body with lots of content"
date_created = xmlrpclib.DateTime(datetime.datetime.strptime("2009-10-20 21:08", "%Y-%m-%d %H:%M"))
categories = ["somecategory"]
tags = ["sometag", "othertag"]
data = {'title': title, 'description': content, 'dateCreated': date_created, 'categories': categories, 'mt_keywords': tags}

post_id = server.metaWeblog.newPost(wp_blogid, wp_username, wp_password, data, status_published)
Posted in programming | Tagged , | 10 Comments

Sample Tomcat application

tomcat

Tomcat is the servlet container that is used in the official Reference Implementation for the Java Servlet and JavaServer Pages technologies. The Java Servlet and JavaServer Pages specifications are developed by Sun under the Java Community Process.

We want to add a custom application to Tomcat called article. The first thing we have to do is edit the file conf/server.xml

<Context path="/article" docBase="article" debug="0" reloadable="true" />

Now we have to make sone directories for our application.

mkdir webapps/article
mkdir webapps/article/WEB-INF
mkdir webapps/article/WEB-INF/classes
mkdir webapps/article/WEB-INF/lib

All *.html and *.jsp go into the directory webapps/article. All *.class files specific to our application go into webapps/article/WEB-INF/classes. All *.class files that are libraries to our application *.class files (typically *.jar files) go into webapps/article/WEB-INF/lib.

Assume we want to have a servlet that writes articles. Now we have to create the file webapps/article/WEB-INF/web.xml.

<servlet>
   <servlet-name>ArticleWriterName</servlet-name>
   <servlet-class>ArticleWriter</servlet-class>
</servlet>
<servlet-mapping>
   <servlet-name>ArticleWriterName</servlet-name>
   <url-pattern>/writer</url-pattern>
</servlet-mapping>

Now we have to create the file webapps/article/WEB-INF/classes/ArticleWriter.java and compile it. After restarting Tomcat, we can access the servlet from http://servername:8080/article/writer.

Posted in programming | Tagged , | Leave a comment

Threads in Python

python

A thread, sometimes called an execution context or a lightweight process, is a single sequential flow of control within a program. You use threads to isolate tasks. Each thread is a sequential flow of control within the same program. Each thread runs independently from the others, but at the same time.

We want to run a certain function a specified number of times per second. A class should be given such a function and some parameters, like the maximum number of threads, the number of active threads per second and a total duration.

import threading, time

class Fire:

   def soldier(self, soldier_nr):

      while (1):

         self.work_lock.acquire()

         if (self.still_working):

            self.write_lock.acquire()
            self.nr_active_soldiers += 1
            self.write_lock.release()

            start_time_counter = self.time_counter

            self.function.__call__(self.arguments)

            end_time_counter = self.time_counter

            self.write_lock.acquire()
            self.nr_active_soldiers -= 1
            self.statistics.append((soldier_nr, start_time_counter, end_time_counter))
            self.write_lock.release()

         else:

            break;

   def __init__(self, function, arguments, nr_threads, per_second, duration):

       self.function           = function
       self.arguments          = arguments
       self.nr_threads         = nr_threads
       self.per_second         = per_second
       self.duration           = duration
       self.work_lock          = threading.Semaphore(0)
       self.write_lock         = threading.Semaphore(1)
       self.still_working      = 1
       self.nr_active_soldiers = 0
       self.time_counter       = 0
       self.statistics         = []

   def fire(self):

       print 'Start'

       for i in range(self.nr_threads):
          thread1 = threading.Thread(target=self.soldier, args=(i, ))
          thread1.start()

       for second in range(self.duration):
          for i in range(self.per_second):
             self.work_lock.release()
          self.write_lock.acquire()
          if (len(self.statistics) > 0):
             (soldier_nr, start_time_counter, end_time_counter) = self.statistics[-1]
             delta_time = end_time_counter - start_time_counter
          else:
             delta_time = -1
          print 'Active', self.nr_active_soldiers, 'Time', delta_time
          self.write_lock.release()
          time.sleep(1)
          self.time_counter += 1

       self.still_working = 0
       for i in range(self.nr_threads):
          self.work_lock.release()

       time.sleep(5)

       output_file = open('output.txt', 'wb')

       for statistic in self.statistics:
           output_file.write(str(statistic) + 'n')

       output_file.close()

       print 'End'

if (__name__ == '__main__'):

   def waste_time(arguments):

       time.sleep(2.5)

   nr_threads = 20
   per_second = 3
   duration   = 15

   firing_squad = Fire(waste_time, None, nr_threads, per_second, duration)
   firing_squad.fire()
Posted in programming | Tagged , | Leave a comment

Sockets in Java

java

TCP provides a reliable, point-to-point communication channel that client-server applications on the Internet use to communicate with each other. To communicate over TCP, a client program and a server program establish a connection to one another. Each program binds a socket to its end of the connection. To communicate, the client and the server each reads from and writes to the socket bound to the connection.

Server

We want to have a client and a server program that can communicate with each other through a socket. The server binds itself to a port number (in our case 1234) and waits for a client connection.

import java.io.*;
import java.net.*;

public class SocketServer {

   int port;

   ServerSocket server = null;
   Socket client = null;

   BufferedReader in = null;
   PrintWriter out = null;

   String input;

   public SocketServer(int port) {
       this.port = port;
   }

   public void listenSocket() {
      try {
         server = new ServerSocket(port);
      } catch (IOException e) {
         System.err.println("Could not listen on port " + String.valueOf(port));
         e.printStackTrace();
         System.exit(-1);
      }

      System.out.println("Opened port " + server.getLocalPort());
      try {
         client = server.accept();
         in = new BufferedReader(new InputStreamReader(client.getInputStream()));
         out = new PrintWriter(client.getOutputStream(), true);
      } catch (IOException e) {
         System.err.println("Could not accept on port " + String.valueOf(port));
         e.printStackTrace();
         System.exit(-1);
      }

      InetSocketAddress clientAddress = (InetSocketAddress)client.getRemoteSocketAddress();
      System.out.println("Connection with client " + clientAddress.getHostName() + " on port " + clientAddress.getPort());
      try {
         input = in.readLine();
         out.println("Pong");
         System.out.println("Input from client: " + input);
      } catch (IOException e) {
         System.err.println("Could not read from port " + String.valueOf(port));
         e.printStackTrace();
         System.exit(-1);
      }

      try {
         client.close();
         server.close();
      } catch (IOException e) {
         System.err.println("Could not close port " + String.valueOf(port));
         e.printStackTrace();
         System.exit(-1);
      }
      System.out.println("Closed port " + String.valueOf(port));
   }

   public static void main(String[] args) {
      SocketServer socketServer = new SocketServer(1234);
      socketServer.listenSocket();
   }
}

Client

The client connects to the server by specifying the hostname and port number of the server (in our case localhost with port 1234).

import java.io.*;
import java.net.*;

public class SocketClient {

   String host;
   int port;

   Socket socket = null;

   BufferedReader in = null;
   PrintWriter out = null;

   String input;

   public SocketClient(String host, int port) {
      this.host = host;
      this.port = port;
   }

   public void listenSocket() {
      try {
         socket = new Socket(host, port);
         out = new PrintWriter(socket.getOutputStream(), true);
         in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
      } catch (Exception e) {
         System.err.println("Could not connect to server " + host + " on port " + String.valueOf(port));
         e.printStackTrace();
         System.exit(-1);
      }

      try {
         out.println("Ping");
         input = in.readLine();
         System.out.println("Input from server: " + input);
      } catch (IOException e) {
         System.err.println("Could not write to or read from server " + host + " on port " + String.valueOf(port));
         e.printStackTrace();
         System.exit(-1);
      }
   }

   public static void main(String[] args) {
      SocketClient socketClient = new SocketClient("localhost", 1234);
      socketClient.listenSocket();
   }
}
Posted in programming | Tagged , | Leave a comment