grayson_o

October 31, 2017
by grayson_o

Resources and Materials

Project Title: Vision Systems and Campus Automation

Project Scope: Classroom Attendance and Intra-campus Transport

Project Duration/timeline: 2016-2017 school year, possibly into 2018 (autonomous golf cart subproject)

Project partners (internal and external): Ilan Naibryf, partner for golf carts.


———————————Resources———————————


Materials (tools, equipment, supplies, locations, remote resources-be specific):


Automated Attendance System

  • Raspberry Pi 3 (rpi3-oliver-evil (pi)@10.14.254.6)

  • USB webcam

  • Phidget RFID reader

  • RFID cards

  • Eventually, access to the current HPA student attendance database

  • Duplicates of all of these, when large-scale distribution is possible


Autonomous Golf Cart

  • “Black box” computer (debian-zen (root)@10.14.4.177)

  • Phidget motor controllers

  • 2 High-torque Pololu servos

  • Arduino Uno

  • Arduino GPS HAT

  • USB webcam

  • Small Elab golf cart


People (include response email from each resource person, agreeing to your materials/support request):

Ilan Naibryf:








Online resources (web pages, wikis, online programs):


Proposal (latest version)


Abstract: This project will focus on first creating an automated attendance system using a combination of Radio Frequency Identification (RFID) tags and facial recognition software. Once such a system has been implemented, the project will branch out into using the vision system for other purposes, such as an autonomous golf cart.



Project Name: Vision Systems and Campus Automation



Background: Working on this project, I would like to create an automatic attendance system that uses a combination of RFID tags and facial recognition with the Raspberry Pi. Initial prototypes of the RFID system for attendance have already been created by previous students such as Erin Evans and Miller White. I may use some of the hardware that they used, as it would be relatively simple to repurpose it for the system I have created.



Purpose: By the end of this project, I expect to have a working prototype of the RFID attendance system that can be deployed to select classrooms at this school. Some locations will also include a camera to improve accuracy via facial recognition. Additionally, I would like to have made significant progress on my autonomous golf cart, having all of the hardware in place with at least a remote-controlled option.



What will you measure: I will teach my computers to identify people and other objects in the world with machine learning and image manipulation software.



How will you measure it: I will test my programs by deploying them in various classrooms, including the Monlab Independent Science Research class.



Resources needed: For the attendance devices, my current prototype uses a Raspberry Pi 3, a USB webcam, and an RFID reader Phidget. Creating any more of these devices will require similar resources, although the Pi-specific camera “Picam” may be substituted to reduce price and more unobtrusive. For the autonomous golf cart, I will require the main brain of the robot - on of the “black box” computers running Debian; an Arduino connected to a GPS module; two high-torque servos, such as the one currently in the remote-controlled boat; a mechanism for controlling the accelerator; and a camera for course correction and visual recognition of hazards. I will work with several other people on this project as well: Zoë McGinnis would like to work on the facial recognition and attendance system, while Ilan Naibryf has agreed to work on the golf cart subproject with me.



Impact: The attendance system will enable more class time to be used on actual class and not on attendance done by the teacher. If attendance is not something they need to consider, then the first 5 minutes of class can be spent on teaching, instead of the human teacher’s valuable time being wasted on making sure all the students are there first. Time may be saved in other areas of campus as well, such as check-in for boarding students - the process could be made more expedient (and thus more likely for all to participate) with an automated system. Instead of requiring central meetings and delays interrupting students' work or free time, a reliable video/RFID system with instantaneous updates updates would be much more effective.


In a similar vein, the autonomous golf cart will save students time, as students will no longer need to spend their off periods or their time between classes to drive injured students around; an autonomous golf cart is also incapable of getting sidetracked or forgetting to pick someone up. If there were enough autonomous carts, the system might even become so efficient that students who were not injured could benefit from it as well, if they had a long walk to the other end of campus or somesuch.



Legacy: At the end of this project, a working automation system will be in place for attendance. While it is up to the school whether or not to use it in the long run, it will be available. Additionally, future students in Independent Science Research will be able to improve the system and make it more accurate, or faster, or change it to be more effective in some way I have not thought of.


With the golf cart, I expect to continue the project next year, as I do not anticipate having my final model finished by the end of the year. Once I graduate, the project will be available to any other student who wishes to improve its artificial intelligence, or has ideas for replicating the project or making it in some way more useful to the school.


0 comments

October 31, 2017
by grayson_o

Proposal Abstract 2016-2017


Abstract: This project will focus on first creating an automated attendance system using a combination of Radio Frequency Identification (RFID) tags and facial recognition software. Once such a system has been implemented, the project will branch out into using the vision system for other purposes, such as an autonomous golf cart.


Project Name: Vision Systems and Campus Automation


Background: Working on this project, I would like to create an automatic attendance system that uses a combination of RFID tags and facial recognition with the Raspberry Pi. Initial prototypes of the RFID system for attendance have already been created by previous students such as Erin Evans and Miller White. I may use some of the hardware that they used, as it would be relatively simple to repurpose it for the system I have created.


Purpose: By the end of this project, I expect to have a working prototype of the RFID attendance system that can be deployed to select classrooms at this school. Some locations will also include a camera to improve accuracy via facial recognition. Additionally, I would like to have made significant progress on my autonomous golf cart, having all of the hardware in place with at least a remote-controlled option.


What will you measure: I will teach my computers to identify people and other objects in the world with machine learning and image manipulation software.


How will you measure it: I will test my programs by deploying them in various classrooms, including the Monlab Independent Science Research class.


Resources needed: For the attendance devices, my current prototype uses a Raspberry Pi 3, a USB webcam, and an RFID reader Phidget. Creating any more of these devices will require similar resources, although the Pi-specific camera “Picam” may be substituted to reduce price and more unobtrusive. For the autonomous golf cart, I will require the main brain of the robot - on of the “black box” computers running Debian; an Arduino connected to a GPS module; two high-torque servos, such as the one currently in the remote-controlled boat; a mechanism for controlling the accelerator; and a camera for course correction and visual recognition of hazards. I will work with several other people on this project as well: Zoë McGinnis would like to work on the facial recognition and attendance system, while Ilan Naibryf has agreed to work on the golf cart subproject with me.


Impact: The attendance system will enable more class time to be used on actual class and not on attendance done by the teacher. If attendance is not something they need to consider, then the first 5 minutes of class can be spent on teaching, instead of the human teacher’s valuable time being wasted on making sure all the students are there first. Time may be saved in other areas of campus as well, such as check-in for boarding students - the process could be made more expedient (and thus more likely for all to participate) with an automated system. Instead of requiring central meetings and delays interrupting students' work or free time, a reliable video/RFID system with instantaneous updates updates would be much more effective.

In a similar vein, the autonomous golf cart will save students time, as students will no longer need to spend their off periods or their time between classes to drive injured students around; an autonomous golf cart is also incapable of getting sidetracked or forgetting to pick someone up. If there were enough autonomous carts, the system might even become so efficient that students who were not injured could benefit from it as well, if they had a long walk to the other end of campus or somesuch.


Legacy: At the end of this project, a working automation system will be in place for attendance. While it is up to the school whether or not to use it in the long run, it will be available. Additionally, future students in Independent Science Research will be able to improve the system and make it more accurate, or faster, or change it to be more effective in some way I have not thought of.

With the golf cart, I expect to continue the project next year, as I do not anticipate having my final model finished by the end of the year. Once I graduate, the project will be available to any other student who wishes to improve its artificial intelligence, or has ideas for replicating the project or making it in some way more useful to the school.

0 comments

October 16, 2017
by grayson_o

Daily Weblog 10/16/17

Today, we went over our goals for Quarter 2. I seem to have ended up with five separate projects:
  1. Security and penetration testing (next steps: Duckuino and possibly ARP cache poisoning)
  2. Hydroponics sensors for the new setup
  3. Moonbase project continuation (and possibly the NASA Ames space colony building competition)
  4. Attendance project and integration with Daniel Mark and others' projects
  5. Generally helping people with their projects, especially Matt Pouchain with learning Python

0 comments

May 20, 2017
by grayson_o

Yearly Summary 2016-2017

This year, I originally proposed to work on "Vision Systems and Campus Automation", with a specific focus on "Classroom Attendance and Intra-campus Transport". I was forced to abandon the transportation side of the project early in the year, when it became clear that the cost of maintaining autonomous golf carts, as well as dealing with the liability associated with autonomous vehicles, was infeasible.

However, I completed a prototype of the campus attendance vision system, and I have plans to continue the project into next year, with a distributed cloud computing solution that may allow full-scale implementation. I did not collaborate with any other students on this project; however, I have left a great deal of documentation and continuation potential for any student who wants to use the OpenCV vision library, and next year I will be working with the Amazon Web Services program Rekognition, for which I will also provide documentation for the ease of use of the next generation of ISR students. Were I to describe this project to someone without any advance knowledge, I would point out specifically the challenges of managing the memory requirements for a large group of people to be recognized. I am very proud of being able to mesh the several different technologies required for the project - from the actual vision system Python libraries, to the MySQL database of users, to the Phidget RFID reader and the webcam - to work as a cohesive whole. I think that I would like to have learned about programs like Rekognition earlier, so that I could have finished a more complete prototype by this point; cloud computing seems like an obvious solution, looking back.

Additionally, I added a new project to the scope of my work that was not included in my initial proposal - creating a hydroponic sensor system to monitor the vital statistics of the plants in the hydroponic column, including electrical conductivity, pH, and temperature. In this project, I collaborated with Sameer Maragh, Sneha Nair, Joshua Bramwell-Butcher, and Minh Truong. This project leaves a very obvious legacy for others in the form of the garden that we have engineered - and my specific contribution to it, the sensor system, which is a platform that can easily be duplicated and built upon by other people, adding more sensors and creating identical systems to service other hydroponic implementations. Next year, I will do exactly that, as we expand the hydroponics program at the Energy Lab. Were I to describe this project, I would focus on the scientific knowledge that I obtained working on the project, such as the concept of nutrient levels in water being described by the electrical conductivity, and the composition of plants determining their ideal range of EC and pH. The greatest challenge in this project was probably creating a sensor system robust enough to survive outside in a wet environment - my previous projects in this class have needed no such hardening of the systems. Were I to do it again, I would have payed more attention to the failings of the system - if I had introduced warning protocols earlier, Dr. Bill would have been notified of the unsafe EC levels on Sunday the 14th, the day many of the plants in the column wilted.

0 comments

May 17, 2017
by grayson_o

Daily Weblog 5/17/17

Today, I presented my project to Ms. Petteys. I think that the presentation went well - I explained my progress throughout this year, with the failed golf cart project, the prototypical attendance project (and its new directions into cloud computing next year), and my current hydroponic monitoring system. I presented a graph of the data, marking major divergence points and events such as the decline due to bubble formation and the water input jumps. The graph is included here, showing the data for the last 2 weeks, which has been an almost completely contiguous data collection period (excluding last weekend, when the program crashed).


0 comments

April 22, 2017
by grayson_o

Weekly Weblog 4/22/17

This week, I reassembled the old PCR box and did research for several projects, while waiting for my last hydroponic sensor to arrive. It has now arrived, so I will finish the hydroponic sensor system next week Monday, and test it for the rest of the week, making sure I can counteract the accuracy-reducing effects of water polarization by timing sensor readings properly. Once that system is working satisfactorily, I will return to the facial recognition program and make sure it iw in working order for my presentation in two weeks. If I have time, I will also develop a prototype using Amazon Web Services program "Rekognition".

0 comments

April 19, 2017
by grayson_o

Daily Weblog 2/6/17

Today, I finished creating and uploading the models for my entire class. I ran out of RAM again, however. I discovered by analyzing the Python memory usage that each user costs 40MB - I will not be able to fit large classes in to one Pi3, even if I were to take steps such as using a compiled language to reduce overhead. I was able to get the program to run (sans Johnny) by opening it again in the console-only mode, which saves operating system memory usage, and will use this mode to test the project as an attendance system for ISR B. For the project to continue, however, I will need to find a way to add RAM.

0 comments

April 19, 2017
by grayson_o

Daily Weblog 2/1/17

I fixed the memory problem! Or at least I hope so. With today's modifications to my project, I was able to cut the weight of the Python script by about 100MB. Additionally, it turns out that I misunderstood how the "free" command works on Linux, neglecting to account for cached memory. It also seems that the full ~100MB of XML data for each model isn't actually necessary, once it has been loaded. I spent the rest of class adding classmates to my database; on Friday I will have them all, and be able to run full-scale tests.

0 comments

April 19, 2017
by grayson_o

Daily Weblog 9/12/16

Today, I worked on the dependency issues between Phidgets and OpenCV. I have resolved a host of issues, and I am now down the last issue of the device not being recognized as plugged in.

An EXTREMELY IMPORTANT NOTE to future raspberry pi/Phidget users! The Phidgets source code has to be edited if you use Python 3, because the sys function that gets the OS version returns "linux2" in Python 2.7, but in Python 3.4 it returns "linux"! Thus, you will have to go into PhidgetLibrary.py and Phidget.py, and edit them to say "linux" instead of "linux2", or else you will get "Platform Not Supported" errors.

0 comments

April 19, 2017
by grayson_o

Daily Weblog 9/9/16

Today, I spent most of my time fixing version differences between the various libraries I have been relying on for my project; the Phidgets library turns out to use Python 2.7, while OpenCV and pymysql use Python 3.4, so I had to edit some of the Phidgets source code and then install it into my Python 3.4 path. All imports are now recognized. While my code will run now, the version using all three libraries crashes still, but that is a fixable issue in the code itself rather than its dependencies. Next class, I will fix those issues.

0 comments

April 19, 2017
by grayson_o

Daily Weblog 8/29/16

Today, I worked exclusively on the attendance database problem. I have managed to make the Python script capable of posting information to the database using the module pymysql (as the default MySQL module was not continued for Python 3.4). I ran into a strange issue where the auto-increment memory in the database was increasing, but new table rows were not being added, but I fixed it; the issue stemmed from having an incorrect field name in the query that my script executed. I need to think of a way of getting the proper student ID values from the table, so that I can add them to the check-in query.

0 comments

April 19, 2017
by grayson_o

Daily Weblog 1/20/17

Today, I planned the database architecture that I will be using for the Attendance project. I set up the MySQL server, made it available over the network (Sequel Pro allows me access from my MacBook), and created tables according to this (albeit incomplete) schema:

Next class, I will integrate the database with the recognition program, so that it can pull a user list from the database for each class and match users to their ID numbers without my having to hardcode the numbers in.

0 comments

April 19, 2017
by grayson_o

Daily Weblog 2/15/17

Today, I tested my program for attendance again, using only my card; despite adding in more print() statements for it to catch errors, the program failed to post anything to the database. The last log statement is that it is beginning the RFID recognition program - no input is ever received from this. The program may be crashing, silently, in that section.


In other news, I have started learning regular expressions, because I thought it might be fun.

0 comments

April 19, 2017
by grayson_o

Daily Weblog 4/19/17

Today, we practiced presenting our final products, and talked about what that would entail, as well as presenting to three visitors. I will need to sign up for as late a presentation date as possible so as to have functional products for each of my projects - I think that I can manage a functional prototype system for both the hydroponics project and the attendance system by then (two weeks from now). The final sensor came in this morning:
I will finish the hardware for the hydroponic sensors on Friday, connecting the pH meter. I will also run tests of how often each can be polled without polarizing the water - this may take until next week.

I also learned yesterday of a system using Amazon Web Services' Rekognition software that might enable me to completely rework the facial recognition for attendance system by outsourcing all of the processing power to the cloud. It was used by the group Sturdy to make a lighthearted project to shoot their CEO on sight with a NERF gun controlled by a Raspberry Pi, but their code is on GitHub at https://github.com/sturdycloud/sting#rekogntion-from-amazon-web-services, and seems trivially adaptable to my system (though I may leave out the NERF gun component). I will look at integrating Amazon Web Services after I have completed the sensor system for hydroponics. Fundamentally, Rekognition allows the user to submit lightly-processed images to pre-constructed cloud databases for fast analysis on Amazon servers.



As with all of Amazon Web Services, a "Lambda Function" is used to submit and receive data quickly, after detecting (for example) the presence or absence of faces. I can use this to quickly analyze the faces of students more accurately than the Pi could on its own, and without running into barriers in how much data can be accessed at once.

0 comments

February 20, 2017
by grayson_o

Daily Weblog 2/20/17

Today, I presented my project to a new prospective student, and researched alternate platforms for the attendance system. In looking at other peoples' C facial recognition programs, however, I came across regular expressions, and decided to learn how to use them; I spent most of class on this.

0 comments

February 18, 2017
by grayson_o

Weekly Weblog 2/18/17

This week, I worked on several side projects, as well as (unsuccessfully) trying to diagnose why my attendance program gives no feedback when it encounters errors; despite rigorous error catching print statements, it seems to crash silently. I think that before proceeding, switching platforms (as I will have to do anyway) may prove useful, as that will conclusively rule out a memory issue.

0 comments

February 10, 2017
by grayson_o

Daily Weblog 2/10/17

Today, I ran the first live test of my attendance program - I had everyone sign in with their cards However, it did not work - there are no entries in the database. What is troubling is that the error logging worked perfectly in that there is a record of the program working, but there are no errors; it did not run out of memory or fail somehow, but also did not provide any meaningful feedback on why it did not recognize any of my classmates.

While the test was running, I used Fresnel lenses to set various things on fire, or melt them - including a aluminum can, a rock, and several pieces of wood. We managed to burn a hole completely through the can.


0 comments

February 8, 2017
by grayson_o

Daily Weblog 2/8/17

Today, I set up a crontab that will activate my recognition program every 10:00 AM on Mondays, Wednesdays, and Fridays, and modified the code to run for 25 minutes instead of a set number of recognized users. I also attached the camera and RFID reader to the Monlab door, so that anyone who comes into the room can sign in using their cards and facial recognition. The first version test has officially started!

0 comments

January 30, 2017
by grayson_o

Daily Weblog 1/30/17

My project works - as of today, I was able to test it in its entirety, despite losing a Pi to water damage (it has since been replaced - thanks, Dr. Bill!). However, I do not have enough memory on the device to actually run the program in its entirety. Of the 1 GB, around 500-600MB are taken up by running processes (I will try to cut down on these at runtime), while the current system of 5 faces takes up almost all of the remaining memory - when I add more people, up to the maximum class size of 20-30 people, I will almost certainly run out of memory. By using only 5 faces to construct models, however, I have been able to reduce the overhead that adding each user creates. I have also modified my Python script to close the database connection when not in use, and I am considering removing the Threading part entirely, relying only on a series of subprocesses to manage my RFID connection. If I am still left without enough memory, I may have to switch devices (the Odroid, apparently, has 16GB of RAM - far more than enough), or port my code to a compiled language, like C++, to save on the interpreter overhead costs that are inherent to using Python.

0 comments

January 27, 2017
by grayson_o

Supplemental Weblog: Git

I have backed up all of my code for the HPA Attendance project in a Git repository, both locally and on GitHub here.

0 comments

November 19, 2016
by grayson_o

Weekly Weblog 11/19/16

This week, I have been reformatting and setting up for my final test of the attendance system. My Raspberry Pi has all of the necessary tools installed, so that when I return to class next week, I will be able to test the system in its final version. First, though, I will need to find a larger SD card, as all of my 8GB has been used.


0 comments

November 19, 2016
by grayson_o

Daily Weblog 11/18/16

Today, I installed OpenCV on my Raspberry Pi. Installing all of the library components takes an absurdly long time, but works reliably when following this guide: http://www.pyimagesearch.com/2016/04/18/install-guide-raspberry-pi-3-raspbian-jessie-opencv-3/. Next time I am in the Energy Lab, I will test the program I wrote combining the RFID reader and OpenCV.

I also worked on a new minor side project; I wrote a Python script designed to work alongside the Arduino piezo buzzer speaker systems that Mr. Halstead's Computer and Electrical Engineering class designed and built. The script can convert any MIDI file into a format that is accessible to the Arduino, although it is not finalized (I plan to finish it on Monday after sports). The code is available on GitHub at https://github.com/OliverGray/MusicConverter.

0 comments

October 26, 2016
by grayson_o

Daily Weblog 10/26/16

Today, I researched how to implement the solution I came up with for the attendance project. Using multithreading and the Python subprocess module, I have successfully built a test bench program that can run other scripts and record their output. I will use these modules to link the OpenCV library to the pymysql and Phidget libraries next class.

I also presented my mid-quarter checkup to Mr. Ferrell and Ms Petteys - I talked about the progress I have made on both the attendance system and my minesweeper robot, rather similarly to my end-of-quarter video.

0 comments

October 25, 2016
by grayson_o

Daily Weblog 10/24/16

Today, I worked on my attendance project - I coped all of my code and project data onto my laptop and then erased the SD card that is the main memory for my Pi (replacing it with a standard NOOBS install), as Phidgets is still not working; I could not uninstall it properly because it was not originally installed using a normal package manager.

0 comments

October 22, 2016
by grayson_o

Daily Weblog 10/21/16

Today, I worked on my attendance system - I reinstalled Phidgets for Python 2. I think that if I reinstall everything, keeping the versions separate, it may be possible to have a central script that runs the other and get input from them independently, thus enabling both the Phidgets and OpenCV libraries to work.

0 comments

October 20, 2016
by grayson_o

Daily Weblog 10/19/16

Today, I updated my Arduino drivers, which did nothing to prevent my laptop crashing when the mBot is plugged in. However, when connected to a computer without the Arduino software, it did not crash - I may try uninstalling all of the Arduino software completely and reinstalling. I also worked for a while on my RFID recognition project - I think I may have a solution to the version issues; if I separate the RFID and facial recognition programs, then call them from a central module, I will be able to get past the problem.

0 comments

October 5, 2016
by grayson_o

Quarter 1 Summary Video



0 comments

October 1, 2016
by grayson_o

Weekly Weblog 9/24/16

This week was spent transitioning from the golf cart project into the new minesweeper project. I did not make significant progress on either of my main projects due to two days of presenting the projects to visitors.

0 comments

September 17, 2016
by grayson_o

Daily Weblog 9/14/16

More dependency issues have arisen. It turns out the "linux2" problem exists in every major file in the Phidget library. I have modified those necessary files, and all of the drivers are now properly found. However, the Pi fails to attach the Phidget object or recognize that it is there - I am not sure of why this is, as the driver works properly and the hardware has also been tested.

0 comments

September 11, 2016
by grayson_o

Weekly Weblog 9/10/16

This week (a short week, so with only two ISR class periods), I worked on integrating the two separate attendance systems that I have built. The RFID system and the facial recognition system both work well, and can post to the MySQL database I have built, so I have been working on creating a single program that recognizes faces when it detects an RFID, and then reports to the database if the two identities match. Due to conflicting versions in the dependencies (until Friday), the program has not been tested much yet.

0 comments

September 9, 2016
by grayson_o

Daily Weblog 9/7/16

Today, I reorganized my workstation to face only a single direction instead of taking up the entire corner - this leaves more space for the CRISPR project, when materials for such arrive.


I also worked on my attendance system code, and I have an initial (untested) prototype of the combined RFID and facial recognition system: when an RFID card is read, the facial recognition system will attempt to recognize the person in front of it, and only write to the database if the two forms of verification match. Next class, I will test the prototype, as it is unlikely to work perfectly the first time.

0 comments

August 28, 2016
by grayson_o

Weekly Weblog 8/27/16

This week was my first week of Independent Science Research - I only had two classes because I switched into this class. I wrote my proposal and began initial work on both sections of my project: fixing the golf carts into a functional state and setting up the beginning of a functional attendance system using MySQL databases.

0 comments

August 26, 2016
by grayson_o

Daily Weblog 8/26/16

Today, I finished organizing my area, added functionality that should allow me to write to the database with facial recognition and check users off the list (although it is so far untested), and worked on troubleshooting the golf cart.


The golf carts have several issues. The charger that was being used on the small golf cart does not, in fact, work - we replaced it with another one that Dr. Bill made. The voltage readout was not working, so we disconnected it. We also removed most of the control panel that our predecessors built. The larger cart is mostly in working order (although Dr. Bill will not let anyone drive it yet), but the smaller cart will require a large amount of work.

0 comments

August 24, 2016
by grayson_o

Daily Weblog 8/24/16

Today was my first day of Independent Science Research this year. I spent most of class organizing my workspace and making sure everything was still there, and then writing the first draft of my proposal abstract. After I got home, I accessed my Raspberry Pi over SSH to make sure that it could still be connected to through the VPN, and made a database for my attendance project.


0 comments

Filter by Tag