Activity Detection Through Mobile App And Hardware Device

Project Goals

Activity recognition is a process identifying the position and activity of the device using an application. Application can be computer based or chip based. Mobile based applications also can be used to operate small hardware devices to sense the signals of the activity and thus identify the activity of the device. The device location like in a vehicle, on a bicycle, on foot, moving and etc. These applications are called as activity tracking app. This app automatically records any activity the device does like walking, cycling and running and etc. Activity recognition is used in some of the famous web based social networking sites like Facebook. Some of the vendors are offering devices like wearable bands. One app in mobile can be used to monitor the daily moves, energy spent and other parameters.  

The aim of the project is to design a mobile app in suitable technology that does activity detection with the help of a hardware device used for sensing the motion. The hardware device will be placed around the wrist. The signals will be passed to the mobile app. Mobile app will be having the control on the sensing hardware device like switching on, operating, configuring and switching off. With the help of a server, the mobile app will identify the activity and does logging. 

Suitable hardware device will be selected for sensing the movement.

Mobile app will be developed

Database design, server design will be done

Suitable algorithms will be used for engineering tasks.

Suitable logging method will be used to log the activity 

The accelerometer scope is given below (Deitel & Deitel, 2015) (Thompson, 2010)

• Accelerometer will be a wrist own
• It senses the limp movement and sends the signal data to the mobile app
• It sends data in real time  

• Mobile App collects the continuous signals from the accelerometer
• Mobile App will be installed in a mobile device
• Mobile device will be connected with the accelerometer
• Mobile app can configure the accelerometer that does the data generation based on the limp movement.
• It captures the signal
• Mobile app algorithm will be doing the following
  • Take the mobile activity as an input
  • Extract the features of the activity
  • Apply feature selection techniques
  • Uses ANN model
  • Uses machine learning algorithms for training and testing the ANN model
  • Identify the activity 

Mobile App may make use of the CMMotionActivity class 

The uses of CMMotionActivity class is given below (Yoon, Park & Chang, 2015). 

Server collects the data sent by the mobile app

Server runs HTTP/FTP application to receive the data from the mobile app

Opens log file, fill the log file and stores the log file 

Python is used in Mobile App development

MATLAB is used for demonstrating ANN modelling, training and testing 

Activity Recognition is used to identify the actions and goals of more than one agents from a series of the agent’s action and environmental conditions. Computer can takes an input as raw data and recognizes the motion activity of user. 

Mobile Sensors are mainly classified into 3 types. They are

• Video sensors
• Environment sensors
• Wearable sensors

In this mobile sensor we can see some sensor which is used in mobile phones.

Accelerometer Scope

It is one of the important mobile sensors in Activity Recognition. At moment of time, an accelerometer can measure acceleration along 3 spatial axes (Thompson, 2010) (McDonald & Stigborg, 2018). 

Compass begins from 0 which is indicates north. The actual reading is, in clockwise the angle between north and current heading direction (Tran, 2016) (Kling & Ellis, 2010).  

It measure the rotation in rad/s. It works along 3 axes i.e. x, y and z (Zhang, Zhang & Wang, 2017) (Jackson Kimball, 2018).

The core techniques are given below

Data Collection

Data Preprocessing

Feature Consumption

Classification 

Data Collections have some Sensors (Vogt, 2010). They are given below

Single type sensor

• Single accelerometer
• Multiple accelerometer 

Multi-modality 

• Different combinations

Locations

• Single location
• Multiple location 

The below picture will give some details about data collection. 

The Preprocessing is also known as DE-noising.DE-noising have Signal filter and Average.

The Feature computation is Time-Domain features and Frequency Domain features. Time-Domain features are Mean, Standard Deviation, Variance, Max, Min, Signal-Magnitude Area and Correlation. Frequency Domain Features are Energy. Time between peaks and Entropy i.e. normalized information entropy of discrete FFT component 

Feature Classification are mainly two types. They are

• Base-level Classification
• Meta-level Classification 

The Base-level Classification are given below.

• Decision Table
• Support Vector Machine (SVM)
• K-Nearest Neighbor (KNN)
• Decision Tree
• Hidden Markov Model (HMM)
• Naive Bayes, Gaussian Mixture Model, Artificial Neural Networks etc. 

The other classifiers are given below.

• Gaussian Mixed Model
• Navie Bayes
• Fuzzy Inference
• Artificial Neural Network
• Rule-based Classifier. 

The Meta-Level Classification are explained below.

Voting

Each and every base level classifier gives a vote, the class label receiving the most votes is the final decision.

Stacking

Stacking is learning algorithm. In this algorithm, how to combine the predictions of base-level classifiers (Quirino & Guidote, 2011).

Cascading

Cascading is an iterative process to combine base-level classifiers. It is Sub-optimal compare to others (WANG, LIU & LIU, 2011).

Activity classification

It is beneficial from more than one base-level classifiers.

Feature selection

It is used to choose the most discriminative features subset.  

There are mainly 5 challenges. They are

• Subject Sensitivity
• Energy and resource constrains
• Activity Complexity
• Data Sparsity
• Location Sensitivity 

A comparative study shows detailed the subject Sensitivity in below figure (FANTL & MCGRATH, 2012) 

Energy and resource constrains is continuous sensing, online updating. Adaptive sensing model make adaption to different activities. Algorithm that requires less computing resource. Less computing resources is no frequency domain features + Lower sampling rate 

Activity Complexity is Multi tasks at the same time which is only predict the activity. Activities transition state is use HMM to smooth the transition states and make the transition state as activities. Culture difference is Globalization vs. Localization. Activities that only involves part of the body i.e. watch 

Data scarcity is not easy to get as many variety of training. It is difficult to coordinate people to same experiment setting (Gunkel & Lange, 2017). 

Uses of CMMotionActivity Class

When sensor’s orientation is different, the feature vector for each axis is also different. With the help of Gyroscope sensor reading the conversion of reading is occurred from body coordination to earth coordination (Webber & O’Kelly, 2010). 

Different part of the body’s movement is different in magnitude for a same activity. It may contain different signal information (Webber & O’Kelly, 2010). 

The Applications are mentioned and explained below.

• Elder and youth care
• Localization
• Personal Biometric Signature
• Industry Manufacturing Assisting
• Daily Life Monitor 

Fall detection 

The youth care is infant sleeping monitor and demand prediction and in class children’s ASD stereotypical movement detection. 

Localization is assist GPS which is Signal in city weak and indoor vertical information (Rao, Kunysz & McDonald, 2012). 

The personal biometric signature is exclusive and unique motion pattern. It can be used in malicious ways like learned signature to infer the keyboard typing 

Industry manufacture assisting is to assist workers in their daily job. 

Daily life monitor like fitness status tracking gadget and mobile apps.

The possible research direction is Data capturing, application like traffic mode recognition, remote control and navigation, and finally algorithms like Multi-modal, Multi-person, Dynamics and Multi-position. 

Implementation plan will be decided after finalising the technologies , testing the technologies for the purpose 

The most important thing is to create new Android application. In this application, minimum SDK is 14 and choose default activity. After opening android studio, open build.gradle file and import play services.

compile ‘com.google.android.gms:play-services:8.4.0’ 

Create a new class named as ActivityRecognizedService and extend IntendService. When play services returns the user’s activity, it will sent to IntentService. This permits you to perform your application logic in background. 

Open AndroidManifest.xml for finish setting. Declare ActivityRecognizedService and include com.google.android.gms.permission.ACTIVITY_RECOGNITION permission for application. 

Base structure of application is completed. Next step is connecting to Google play services and activity data requesting. 

For using play service, first connect them. Open MainActivity.java and implement OnConnectionFailedListener and ConnectionCallbacks interfaces. Then create member variable type GoogleApiClient to keep a reference to API client. 

The required interface for GoogleApiClient are implemented. After implementing that, initialize the client and connect Google Play Service in oncreate( ) by requesting ActivityRecognition.API. With the help of GoogleApiClient we can associate the listeners. 

After the GoogleApiClient instance has connected, Call onConnected( ). Create a PendingIntent that goes to IntentService (already created) and pass it to ACtivityRecognitionApi.

In onHandleIntent () method of ActivityREcognizedService, validate the received Intent. It contains activity recognition data. Extract the ActivityRecognitionResult from the Intent to see the performing activities. We can retrieve a list of the ActivityRecongnitionResult object.

Core Techniques

For this application, each activity has been detected. It display how the user is performing that activity by calling getConfidence () on a DetectedActivity instance. Assume the confidence is 75 or greater than 75 means it is safe to user performing activity. To demonstrate this, notification is displayed. It displays the app detect the user is walking with high confidence. 

If the user run this application, go for a run and then plug the device into computer. The following similar log will be show. 

If the user go for a walk, the user can receive a notification like “Are you walking?”. If the user have an exercise application then the user allow to start keeping track of their exercise 

This mobile application is developed for the activity of the device like walking, running etc. This can be done with the help of sensors like video, environmental and wearable sensors. Xml file is used to develop the frontend and SQLite is used to provide the backend connection. Declaration of sensor interval threshold can be done with the help of accelerometermanager java file. Some web view is showed in the gyroscope. The java language is used to connect between frontend and backend. The textbox shown the details like total distance travelled by the person, total steps covered by the person etc. And finally it can analyze the health condition of the person 

Conclusion 

The activity and position of any device can be detected with the help of activity identification is shown. The working of accelerometer sensor is studied with the help of activity recognition. The direction can be detected using compass is studied. The gyroscope detects the rotation is studied. The activity of the device is analyzed with the help of smartphone sensors. The working of hardware devices which is used to sense the motion is described. The core techniques like data collection, preprocessing, feature consumption and classification are studied. The five main challenges like energy and resource constraints, subject sensitivity, data scarcity, active complexity and location sensitivity is analyzed. The various applications like daily life monitor, industry manufacturing assisting, elder and youth care, personal biometric signature and localization are described. 

References 

Deitel, H., & Deitel, P. (2015). Android. Pearson Education UK.

FANTL, J., & MCGRATH, M. (2012). Contextualism and Subject-Sensitivity. Philosophy And Phenomenological Research, 84(3), 693-702. https://dx.doi.org/10.1111/j.1933-1592.2012.00585.x

Gunkel, A., & Lange, J. (2017). Water scarcity, data scarcity and the Budyko curve—An application in the Lower Jordan River Basin. Journal Of Hydrology: Regional Studies, 12, 136-149. https://dx.doi.org/10.1016/j.ejrh.2017.04.004

Jackson Kimball, D. (2018). Spin Gyroscope is Ready to Look for New Physics. Physics, 11. https://dx.doi.org/10.1103/physics.11.5

Kling, T., & Ellis, J. (2010). The compass. New York: Perseus Books Group.

McDonald, M., & Stigborg, P. (2018). Measuring accelerometer dynamic range from seismic data using Allan deviation. Geophysical Prospecting. https://dx.doi.org/10.1111/1365-2478.12616

Quirino, J., & Guidote, A. (2011). Two-step stacking in capillary zone electrophoresis featuring sweeping and micelle to solvent stacking: II. Organic anions. Journal Of Chromatography A, 1218(7), 1004-1010. https://dx.doi.org/10.1016/j.chroma.2010.12.095

Rao, B., Kunysz, W., & McDonald, K. (2012). GPS. Norwood: Artech House.

Thompson, A. (2010). A ballistic filter for GPS and accelerometer measurements. Aberdeen Proving Ground, MD: Army Research Laboratory.

Tran, D. (2016). Editor’s Introduction Compass 13. Compass: Journal Of Learning And Teaching, 9(13). https://dx.doi.org/10.21100/compass.v9i13.430

Vogt, W. (2010). Data collection. Los Angeles, Calif.: SAGE.

WANG, J., LIU, Y., & LIU, X. (2011). Model for Cascading Faults in Complex Software. Chinese Journal Of Computers, 34(6), 1137-1147. https://dx.doi.org/10.3724/sp.j.1016.2011.01137

Webber, M., & O’Kelly, M. (2010). Empirical Tests and Sensitivity Analysis of a Model of Residential and Facility Location. Geographical Analysis, 13(4), 398-411. https://dx.doi.org/10.1111/j.1538-4632.1981.tb00747.x 

Yoon, K., Park, Y., & Chang, N. (2015). A generating method of CM parameters of pairing-friendly abelian surfaces using Brezing-Weng family. Journal Of The Korea Institute Of Information Security And Cryptology, 25(3), 567-571. https://dx.doi.org/10.13089/jkiisc.2015.25.3.567

Zhang, F., Zhang, W., & Wang, G. (2017). Non-driven Micromechanical Gyroscopes and Their Applications. Berlin, Heidelberg: Springer Berlin Heidelberg.