Task 2: Demo Code Package And Technical Report For Integration Plan

Overview of Task 2

RESTful web services are very useful for the different web applications that depends on the multiple data sources on the internet to provide and meet a specific data requirements of the users. For this the data and service integrations is an important aspect.  For GE it is important to successfully integrate the data available from different sources as well as the applications that uses this data.  In addition to that, use of the latest technological trends such as cloud-based services, data analytics (after cleaning) available from different sources and mobile computing to gain competitive advantage in the market and efficiency in its different processes. 

In this paper we will discuss about the techniques used for cleaning and merging of data sources that are in different formats. After this development of a RESTful web service and its use in a mashup application is discussed in the different sections of this report. 

Data merging and cleaning

  Initially we have two data files which are clinics.csv and location.xml. the first one contained the details of the clinics with the postal code and the second one contained the longitude, latitude values of the location of the clinic along with the postal code.

 In order to merge the two data sets, we first converted the xml file in to csv file using the “xml” package available in python. After this step the next step was to merge and lean the data sets.  Depending on the common column (the “PostalCode” column) in both the csv data sets we used the “pandas” data analytics package to merge them. Following is the screenshot of the code used for the cleaning and merging of the data sets.In the merged dataset there are some emails addresses of the clinics are not complete as they do not contain the @myclinic part.  In order do this we found the use of the “Petl” package as used in the lectures and provided in the notes.

After cleaning and refining the merged data set we got the table in a consistent format that can be used as a consistent data set for the RESTful as well as Mashup application. The redefined dataset in the csv files contains the following data, 

Name	Suburb	State	Postcode	Email	Lat	Lon
22	Ipswich Clinic	Ipswich	QLD	4305	[email protected]	-27.614604	152.760876
23	Cattle Creek Clinic	Cattle Creek	QLD	4407	[email protected]	-27.648300	150.836678
24	Caboolture Clinic	Caboolture	QLD	4510	[email protected]	-27.085007	152.951707
25	Booie Clinic	Booie	QLD	4610	[email protected]	-26.498426	151.935421
26	Rockhampton Clinic	Rockhampton	QLD	4700	[email protected]	-23.378941	150.512323
27	Townsville Clinic	Townsville	QLD	4810	[email protected]	-19.267358	146.806540
28	Cairns Clinic	Cairns	QLD	4870	[email protected]	-16.925397	145.775178
29	Adelaide Centre Clinic	Adelaide	SA	5000	[email protected]	-34.925770	138.599732

 RESTful Web services

The Web services are different types of service delivered through the web or internet. Every service ono internet is considered as an application module. These services communicate with each other and other web applications through the open protocols. The protocols through which the services communicate are WSDL and WSGI. These are mainly based on the XML and HTTP.

Data Merging and Cleaning

The traditional SOAP based web services, which are large in size and consumes a lot of bandwidth in delivering the services.  Compared to the traditional SOAP based web services, the REST architecture based web services are much lightweight and small which can be integrated with the any web applications in order to improve the performance of the applications for user’s data requirements.

 In Rest based web services, the resources are represented as the virtual folders on the internet. The services provide accessibility to these folders identified by the URI (Unique resource Identifiers)

 For this assignment, we have developed a web service server, that will get a postal code to get the details of the clinic using the merged data source which is “Clinic_location.csv” RESTful web services use the data caching technique in order to minimize the response time against a user’s data query through the web service. Rather than generating the same results frequently it stores the same results and delivers from the cached results. 

After the webserver service is executed successfully it will occupy a specific port as shown in the image,Mashups

The mashup applications are used in order to provide a unified view of different kind of data collected from different sources having different attributes of data. By using t mashups, the web applications help the end users to get the required data analysed from different perspectives. 

The mashup application developed for this project integrates the data from the merged dataset in the day merger stage of this project. The mashup application is Integrated with the Google maps application to plot the location of the clinics that are given in the csv file clinic_locatiin.csv. after getting the details from the csv dataset it will find the location on the Google map and plot the marker at the specific longitude and latitude.

 In order to get the detailed information about the clinic on the map the application uses ajax calls. 

Demo running instructions

The mashup application or the HTML page can take the postal code of any region of Australia. After this the web server gets the subtle elements for the postal code to plot the area of the clinics on the map.

The information for a particular postal code is gotten as the Json information question. This information question is then devouring by the application and marker is plotted on the google delineate.

For utilizing the application, at first the web application should be executed utilizing any python IDE. In the following stage the HTML page is to be opened and any postal code to check if there is any centre situated in that postal address then the web service will plot a marker on that locations by retrieving the longitude, latitude and other details.

As the RESTful web services can give pre-assemble extensibility to the web applications and administrations and gauges in this way enables the designers to utilize less measure of code with the implicit blunder taking care of highlight for the applications to which these web services are coordinated.

Conclusion

Being lightweight and low in bandwidth consumption the RESTful web services are the obvious choices for the developers to integrate with other applications and make those applications more efficient and productive.

The REST architecture is favoured by the developers rather than traditional SOAP or Simple Object Access Protocol architecture based web services due to its above mentioned features.