Patient Monitoring Smart Wheelchair

As part of my final year project our team chose the topic as Patient Monitoring Smart Wheelchair. We wanted to make something for the society. As a part of one of my earlier projects Improving Infrastructure for specially challenged citizens in Vashi we had conducted a survey. We found out that the existing infrastructure in Vashi(Navi Mumbai, India) is not at all disabled friendly and same is the situation in majority of India. On top of that the modern assistive technology is too expensive for the common man or aam aadmi.

proportion of diabled population

Above figure shows the percentage of disabled population to total population in India as per 2011 census. In states like Maharashtra, Andhra Pradesh etc. the percentage is very high.

type of disability

Above figure shows the proportion of disabled population by type of disability. We can see that disability in movement accounts for 20.3 % and multiple disability 7.9%. This led us to the idea of making a smart wheelchair.

rural vs urban

This shows that a major chunk of the disabled population lives in rural India. Our market analysis led us to the finding that all the modern electric wheelchairs available in the market have outrageous price tags.

market analysis

So we decided that our wheelchair would be affordable. Then depending on the analysis of disabilities we found out which all modes should be incorporated into this smart wheelchair.

The modes selected are :

  • Voice control mode
  • Gesture control mode
  • Joystick control mode

Patient monitoring was thought of as an added service. Initially the plan was to measure heart rate, body temperature and blood pressure. But due to time constraints we limited to just measuring body temperature and displaying it on a LCD.

Here is a short video of the completed project.

The bill of materials is as follows.

bom

As you can see we achieved the cost reduction. So that is all from my side in this post. Thank you for patiently reading this and reaching till the end. If you people want a post about the technical details I will require at least 10 Word press likes Winking smile. Share with as many as you can. Maybe someone will take this idea to the next level. Thank you once again for reading this post.

 

P.S: I just felt like sharing this news article with you all.

http://economictimes.indiatimes.com/news/politics-and-nation/government-set-to-bring-private-sector-under-the-new-disability-legislation/articleshow/48780998.cms

Advertisements

Gesture control for PowerPoint presentation

Basic idea of the project

The intention of this tutorial is to learn how to control computer keyboard and mouse events using your micro-controller. For demonstration I have chosen to name the project as gesture control for PowerPoint presentation. At the end of this tutorial you will be able to control the slides using your gestures. Now in order to control the operating system you need a language like python, java, c++ etc. I find python affable and powerful so I’ll be using that for coding. You can use any language you want but the logic and algorithm will remain the same, only the syntax will differ. I won’t be teaching you python in this post that is beyond the scope of this post. If you want to learn the language there is plethora of content available online. You just need a net connection and off you go.
P.S. : If you like the posts do like and share them with others.

Python

Python is an easy to learn high level programming language. It is a beautiful and a very powerful language. The packages that are available make it kind of limitless. Some of the places where python is used are mentioned below.

  • Google makes extensive use of Python in its web search systems.
  • The popular YouTube video sharing service is largely written in Python.
  • The Dropbox storage service codes both its server and desktop client software primarily in Python.
  • The Raspberry Pi single-board computer promotes Python as its educational language.
  • NASA, Los Alamos, Fermilab, JPL, and others use Python for scientific programming tasks.

So we know that most of the big shots use python. Now they use it for a reason and the reason being that its simply an awesome language. If you want to start learning programming you ought to start with python. Here are a list of sites and books that you may use for learning python.

  1. https://www.python.org/about/gettingstarted/ (This is the official page where you can learn how to install the IDE and get started.)
  2. Learning Python, 5th Edition (This is a good book if you are new to programming and otherwise as well.)
  3. http://www.learnpython.org/
  4. http://www.tutorialspoint.com/python/

Once you get the hang of it then you can directly use the documentations for learning how to use the packages.

pySerial and PyUserInput

We will be requiring these modules in our project. The names are quite self explanatory the former is for serial communication while the latter is for the mouse and keyboard events. The links to these modules are:

Well download these and install them. I recommend you to use 32bit python 2.7 version modules as well as the language. Because most of the modules are available for 2.7 version.

Components and Software requirements

  • A microcontroller board with UART capability e.g. MSP430G2 Launchpad, Arduino Uno board etc.
  • An accelerometer e.g. ADXL335 etc..
  • Python 2.7 , pySerial & PyUserInput modules

Connections

connections

I have used fritzing for making this. Here is the link to their home page.  http://fritzing.org/home/

Logic

We will calibrate the accelerometer and take readings for left and right position. Use the serial monitor for this. Read my tutorial titled Capacitive Accelerometer Interfacing if you don’t know what I am talking about. Next once you have those digital values you need to make the program for slide control. We know that left arrow and right arrow keys are used for navigation purpose. So in our python script the if statements will contain code for left arrow button press and right arrow button press. Note that you are reading the values that the controller is sending serially using python and taking decisions based on that value.

Energia Code

int x_pin = A0;
void setup()
{
  // put your setup code here, to run once:
  Serial.begin(9600);
  pinMode(x_pin,INPUT);
  analogReference(INTERNAL2V5);
}

void loop()
{
  // put your main code here, to run repeatedly:
  int x = analogRead(x_pin);
  Serial.println(x);
  delay(500);
}

Python Script

__author__ = 'MANPREET'
'''
This is a file for controlling keyboard events.
'''
from pykeyboard import PyKeyboard
import serial
import time

comPort = raw_input("Please enter the COM port number")
baudRate = raw_input("Please enter the baud rate")
myserial = serial.Serial(comPort, baudRate)
k = PyKeyboard()
TRUE = 1;
try:
    while (TRUE):
        if (myserial.inWaiting()):
            mydata = myserial.readline()
            x = int(mydata)
            print(x)
            if x > 650:
                k.tap_key(k.left_key)
                print("left")
                time.sleep(1)
            if x < 550:
                k.tap_key(k.right_key)
                print("right")
                time.sleep(1)
except KeyboardInterrupt:
    print("stop")

Code Explanation

The Energia code is pretty straightforward but still just to cover that as well. We have declared A0 i.e. P1.0 pin as input and changed the ADC reference voltage to 2.5V in line 7. Next part is just getting the ADC reading and sending it serially.

The python code demands some explanation. So lets begin understanding the code .

from pykeyboard import PyKeyboard
import serial
import time

This code will import three modules PyKeyboard, serial and time. For the pykeyboard we have imported the constructor. Then you have made one object k using the same.

comPort = raw_input("Please enter the COM port number")
baudRate = raw_input("Please enter the baud rate")
myserial = serial.Serial(comPort, baudRate)
k = PyKeyboard()

myserial is an object of the serial module that you have imported. You will use this to access its functions. The raw_input() is for taking the com port and baud rate values from the user. Example COM11 and 9600.

TRUE = 1;
try:
    while (TRUE):
        if (myserial.inWaiting()):
            mydata = myserial.readline()
            x = int(mydata)
            print(x)
            if x > 650:
                k.tap_key(k.left_key)
                print("left")
                time.sleep(1)
            if x < 550:
                k.tap_key(k.right_key)
                print("right")
                time.sleep(1)
except KeyboardInterrupt:
    print("stop")

Well this is an infinite loop and you are checking this block for keyboard interrupt i.e. ctrl+c . This is done so that you can come out of the program properly without having to kill the program. Next we are checking if there is data in the serial buffer. If yes then we are storing it in mydata variable. Convert it into integer and store it as some variable say x. Next step is easy write two if statements and include the code and condition for left arrow button press and right arrow button press. For more details of the PyKeyboard module visit : https://pypi.python.org/pypi/PyUserInput/0.1.9
For running the python script install python 2.7. Copy paste the python script code into notepad and save it as gersturecontrol.py(or any name for that matter) Then follow these steps.

  1. Open command prompt(Press windows+r, then type cmd and press enter.)
  2. opening_command_prompt

  3. Change the directory to the one containing your python script i.e. the .py file. Use cd for that.
  4. file_location

  5. Use python gesturecontrol.py for running your code
  6. running the program

  7. For stopping the code press ctrl+c

Thank you for reading the post and hope that it was helpful.If you like the post do share it with others and spread the knowledge.

Wireless Notice Board using SIM 300 and EK-TM4C1294XL

Well I just loved the idea of a wireless noticeboard which will display the notice whenever some sms is sent to it. That way you can intimate the concerned people about the notice or information even if you are not physically present at the location. This can be used in colleges wherein two notice boards can be installed per department. One will be used by the principal and the other may be by the head of dept. or the teachers. This is a quick way to send the info. So starting with the list of materials required.

Components

  • Micro-controller board with at least two hardware UARTs(not mandatory but will be beneficial.)
  • SIM 300 module or equivalent(with the charger etc.)
  • Some display e.g. LCD, dot matrix, graphical LCD etc.

Connections

connections

EK-TM4C1294XL Pins

TM4C129-X11-—-Pins-Maps

Code Explanation

The main objective is to extract the message from the received message. Now you can use some special character for this but I’ve decided to use the concepts of html tags. So my tag is <s></s>. Whatever is written in these tags will be displayed. String object provides beautiful string manipulation functions. So we will not reinvent the wheel but use it in our application so to speak. Let’s look at the code that will extract the message from the received data.

  char buffer[250];
  Serial.readBytes(buffer, 250);
  String message = buffer;
  String command = "<s>";
  String commandEnd ="</s>";
  int indexOfMessage = message.indexOf(command);
  int indexOfMessageEnd = message.indexOf(commandEnd);
  if(indexOfMessage>0 && indexOfMessageEnd>0){
  String actualMessage = message.substring(indexOfMessage+3,indexOfMessageEnd);
  Serial.print("Message             :");
  Serial.println(message);
  Serial.print("Command             :");
  Serial.println(command);
  Serial.print("CommandEnd          :");
  Serial.println(commandEnd);
  Serial.print("Actual Message      :");
  Serial.println(actualMessage);
  actualMessage.toCharArray(actualMessageArray,250);

We are making two strings command and commandEnd. These will store our tags. Next we need to find the index of these tags. For this we use indexOf() and this will return –1 if the string is not present. So we need to send the message only if both the indices are not –1.Next is just one toCharArray(), this is for the LCD function. Also the serial printing is just for our reference. You may remove those lines.

The scrolling part is taken from arduino cook book. Here is the link.

Thank you for reading this. If you liked this post do share it with others!!

Entire Code

#include <LiquidCrystal.h>;
LiquidCrystal lcd(42,44,49,50,70, 69);

const int numRows = 2;
const int numCols = 16;
char actualMessageArray[250];

void setup()
{
	  // put your setup code here, to run once:
	  Serial.begin(9600);
	  Serial7.begin(9600);
	  Serial7.println("AT");
	  while(Serial.available()>0)
	  {
		Serial.print((char)Serial.read());
	  }
	  delay(1000);
	  Serial7.println("AT+CMGF=1");
	  while(Serial.available()>0)
	  {
		Serial.print((char)Serial.read());
	  }
	  delay(1000);
	  Serial7.println("AT+CNMI=2,2,0,0");
	  delay(100);
	  while(Serial.available()>0)
	  {
		Serial.print((char)Serial.read());
	  }
	  delay(1000);
	  lcd.begin(numCols, numRows);
	  lcd.setCursor(0,0);
}

void loop()
{
  // put your main code here, to run repeatedly:
 if(Serial.available()>0)
 {
	  char buffer[250];
	  Serial.readBytes(buffer, 250);
	  String message = buffer;
	  String command = "<s>";
	  String commandEnd = "</s>";
	  int indexOfMessage = message.indexOf(command);
	  int indexOfMessageEnd = message.indexOf(commandEnd);
	  if(indexOfMessage>0 && indexOfMessageEnd>0){
	  String actualMessage = message.substring(indexOfMessage+3,indexOfMessageEnd);
	  Serial.print("message          :");
	  Serial.println(message);
	  Serial.print("command          :");
	  Serial.println(command);
	  Serial.print("commandEnd       :");
	  Serial.println(commandEnd);
	  Serial.print("actualMessage    :");
	  Serial.println(actualMessage);
	  actualMessage.toCharArray(actualMessageArray,250);
	 }
 }
  marquee(actualMessageArray);
  delay(1000);
  lcd.clear();
}
void marquee( char *text)
{
	int length = strlen(text); // the number of characters in the text
	if(length < numCols)
	lcd.print(text);
	else
	{
		int pos;
		for( pos = 0; pos <= numCols; pos++)
		lcd.print(text[pos]);
		delay(1000); // allow time to read the first line before scrolling
		pos=1;
		while(pos <= length - numCols)
		{
		lcd.setCursor(0,0);
		for( int i=0; i < numCols; i++)
		lcd.print(text[pos+i]);
		delay(300);
		pos = pos + 1;
		}
	}
}

Serial Monitor Output

notice board

Video

A taste of Image Processing

Image processing involves extracting information from images and using the information so obtained for various operations and tasks. Don’t confuse image processing image processing with image manipulation that involves adjusting the images. Photoshop and similar software are used for image manipulation.

Application areas

  • Medical Applications
  • Industrial Applications
  • Military Applications: Some of the most challenging and performance-critical scenarios for image processing solutions have been developed for military needs, ranging from detection of soldiers or vehicles to missile guidance and object recognition and reconnaissance tasks using unmanned aerial vehicles (UAVs). In addition, military applications often require the use of different imaging sensors, such as range cameras and thermo-graphic forward-looking infrared (FLIR) cameras.
  • Law Enforcement and Security: Surveillance applications have become one of the most intensely researched areas within the video processing community. Biometric techniques (e.g., fingerprint, face, iris, and hand recognition), which have been the subject of image processing research for more than a decade, have recently become commercially available.
  • Consumer Electronics
  • The World Wide Web

So as we can see image processing has wide areas of application.

That being said you need some software or programming language for making this image processing possible. There are various ways by which this can be done. You can use c, NI LabVIEW, MATLAB etc. for image processing. For this post I’ll be using MATLAB. Now we did this just so as to understand the concepts of image processing. So if you feel that its meagre you can build on this.

Code

clc;
close all;
clear all;
video = videoinput('winvideo');%Create video variable

set(video,'FramesPerTrigger',1); % Setting frames per trigger
preview(video);%Preview the video
rgb_image = getsnapshot(video); % Storing Image in an array variable
[y x c]= size(rgb_image); % Determining the size of the captured frame.
x1 = (x/2)-(0.2*x);
x2 = (x/2)+(0.2*x);
y1 = (y/2)-(0.25*y);
y2 = (y/2)+(0.25*y);
global s;
s = serial('COM4')
fopen(s)
while(1)
image = getsnapshot(video);
fR = image(:,:,1);
fG = image(:,:,2);
fB = image(:,:,3);
I = fR>200;
se=strel('disk',5);
B=imopen(I,se);
final=imclose(B,se);
[L,n]=bwlabel(final);
for k=1:n
    [r,c]=find(L==k);
    rbar=mean(r);
    cbar=mean(c);
end
rbar
cbar
if x1<cbar<x2 &&  rbar<y1
    disp('Move forward');
    global s;
    fwrite(s,'w')
elseif cbar<x1 && y1<rbar<y2
    disp('Move right');
    global s;
    fwrite(s,'d')
elseif cbar>x2 && y1<rbar<y2
    disp('Move left');
    global s;
    fwrite(s,'a')
elseif x1<cbar<x2 && rbar>y2
    disp('Move back');    
    global s;
    fwrite(s,'s')
elseif x1<cbar<x2 && y1<rbar<y2
    disp('Move stop');
    global s;
    fwrite(s,'f')
end
end

Now this is the code. I am thinking I’ll just explain the logic and then you can use MATLAB help for the rest. Seriously the help provides is simply awesome. You can understand how the functions word using the help and then there is the world wide net. So I am assuming that those of you who has interest will read further.

Now let’s begin understanding the code.

video = videoinput(‘winvideo’);

This will create a video object from the available cameras. You can check the available ones by using imaqhwinfo

image

So for the windows winvideo is installed adapter.

image

Next give winvideo as the agrument to the imaqhwinfo(). Now if you have an external webcam connected to your computer you will se two device ID’s. So now suppose you want to know about the device with ID=1 all you need to do is pass device ID as second argument.

image

You can see the properties you the webcam.

So just use the videoinput() for creating a variable attached to the particular webcam in MATLAB. In our case video is the variable.

Next you set the frames per trigger i.e. whenever you give the capture commands how many frames will be captured every time.

Then you see a preview of the video feed so that you come to know what exactly is the camera viewing.

Next up you take a sample shot so as to determine the dimensions of the camera.

You will get a 3D matrix of the image. Extract the x and y resolution.

Now for this application what we will do is divide the webcam field into 9 quadrants and take decision depending on position of the image of the object to be detected.

Now since we wanted to learn the basic we used white light such as a torch as the source. If you have a proper webcam and good lighting conditions you can detect normal coloured balls using this program. All you need to do is some thresholding.

Now as usual open the serial port and send a particular character depending upon the quadrant in which the object lies.

So you basically take a snapshot again and again and manipulate that snap. So each image is made of 3 components red, blue and green. Since we are using white light as the object it does not matter which component you choose. But if there is a particular colour that the object has take that particular matrix. Now do the thresholding so that you get only the torch circle on the screen.

Then you remove the noise and coalesce the remaining parts to form a single body.

Then you calculate the centroid and take decision.

image

This is a sample of what the image will look like after thresholding. So as you can see it lies in the middle quadrant.

We had written the code such that it will transmit w,a,s,d,f depending on the quadrant. Do top middle corresponds to w and so on you can figure that yourself.

On the controller side we manipulate the data to control the bot.

I’ll be uploading the video soon. Thank you for reading. Hope this was useful.

GUI controlled bot

Well in this post I’ll be covering how to create a GUI for controlling a bot using MATLAB. Now this concept can be readily extended to various other stuff. As after learning alphabets one can make words and sentences and essays so can one make various applications using basics. So let’s start, shall we?

To begin with you need to read this post on creating GUI because I’ll be assuming you have read that. So we know how to make buttons now. So this tutorial will focus on using this GUI along with serial class of MATLAB.

Serial Communication in MATLAB

Well we all know man is a social animal. We need someone or the other to communicate with others. We may use words or gestures or expressions etc. Similarly MATLAB needs to communicate with the outside world to get data in order to manipulate the data and send the processed result back. The way to communicate is via RS232 protocol.

There is a serial class in MATLAB that lets you to create objects and manipulate its properties.

Syntax:

obj = serial(‘port’)

Here port is the COM name and obj is the object name. You can change the baud rate and other properties like number of party bits and stop bits etc. I won’t be covering that in this post. If you have further interest you may refer to their website.

Now that we have a object for the serial COM port we need to open that port for serial communication. Now the syntax for the same is as follows.

fopen(obj)

This will open the COM port for communication. Similarly there is fclose(obj) that will close the port.

Now that we know how to open and close the port might as well learn how to write data to the port.

fwrite(obj,data)

Here obj is the object created and data is 1 byte data that you want to send/write on the port.

For example :

fwrite(s,’a’)

Well that takes care of the serial part that is required for our task. Remember that internet is a universe of knowledge and you can travel it to unravel new mysteries whenever you like.

Now many of you out there who are programmers are already aware of the concept of global and local variables. We’ll require this concept as well. For creating a global object just type global keyword before the object. Now whenever you want to access the global version of the variable just type global before the object or variable.

Now in the GUI code you need to create a global object, open the port. Then the logic we’ll be using for making this GUI work is as follows:

1) Write code in the button callback function to send a specific character. For example send ‘W’ for forward and so on.

2) Now the controller comes into picture wherein you have to receive the character and take some action depending on the character received.

So basically you need to use the UART module of your controller whether it be msp430 or PIC or 51 or arduino or atmel or and processor for that matter. The logic is simple if you are using embedded c or assembly language you can write a simple ISR that will check the character received serially and send some data to port pins that will drive the motor via the motor driver.

I think this much explanation is enough. If you have any queries you can always mail me or comment, I’ll get back to you as soon as possible.

Code

/*
 * serial_reception.c
 *
 *  Created on: 30-Mar-2014
 *      Author: MANPREET
 *     Website: https://learningmsp430.wordpress.com/
 */

#include <msp430g2553.h>
#include "serial.h"

void main(void)
{
	WDTCTL = WDTPW + WDTHOLD; // Stop watchdog timer
	uart_init(); // Refer the UART post for the functions
	P1DIR |= BIT3|BIT4|BIT5|BIT6;
	P1OUT = 0;
	 IE2 |= UCA0RXIE;     // Enable USCI_A0 RX interrupt
	 __bis_SR_register(LPM0_bits + GIE); // Enter LPM0, interrupts enabled
}

#pragma vector=USCIAB0RX_VECTOR
__interrupt void USCI0RX_ISR(void)
{
 if(UCA0RXBUF=='W' | UCA0RXBUF == 'w')
 {
	 P1OUT |= BIT3|BIT5;
	 P1OUT &= ~(BIT4|BIT6);// Send 1010 for forward
 }
 if(UCA0RXBUF=='S' | UCA0RXBUF == 's')
 {
	 P1OUT |= BIT4|BIT6;
	 P1OUT &= ~(BIT3|BIT5);// Send 0101 for backward
 }
 if(UCA0RXBUF=='A' | UCA0RXBUF == 'a')
 {
	 P1OUT |= BIT4|BIT5;
	 P1OUT &= ~(BIT3|BIT6);// Send 0110 for left
 }
 if(UCA0RXBUF=='D' | UCA0RXBUF == 'd')
 {
	 P1OUT |= BIT3|BIT6;
	 P1OUT &= ~(BIT4|BIT5);// Send 1001 for right
 }
 if(UCA0RXBUF=='F' | UCA0RXBUF == 'f')
 {
	 P1OUT &= ~(BIT3|BIT6|BIT4|BIT5);// Send 1001 for right
 }
}

MATLAB Code link:

https://www.dropbox.com/s/iyddas4ynf93rz5/project_bot.fig

https://www.dropbox.com/s/78fzdopyci2g5km/project_bot.m

GUI:

GUI

 

Note :

The connections are obvious if you read the code. You have to use a driver like L293d of l298. If you face any difficulties doing this feel free to contact me. I’m happy to help.