Swarmathon Project blog

This blog will record updates and meeting notes about the progress of the Robotics team and the internship in general terms.

Reminder: project is part of a competition.  Information regarding code and methodolgy may therefore be described in very general terms since this site is necessarily public.

May 4, 2017

No formal meeting today.  Participants have been previously instructed to review CRA/CREU report guidelines and provide input.  Brittany and Ashlee will focus on providing additional literature review materials for possible inclusion in the final paper.

The entire group thinks we could have accomplished more had we applied more focus and time to the project.  While we did not win the competition, every member of the team has gained valuable experience in team work, goal setting, breaking large problems into smaller ones, responsibility, and accountability.   We have not mastered these concepts.  But we have a greater awareness of our strengths and weaknesses, our successes and our failures.. no, not failures.  Unintended results.  If we learned from our experiences and resolve to improve our performance in the future, then we cannot say we failed.

Short term Robotics Goal: Submit paper by May 5th

Long term Robotics goal: Continues to improve upon current code to serve as a resource for the next team with emphasis on improving localization and implementing the Explore-and-Exploit algorithms.

April 27, 2017

The meeting today focused on ensuring all blogs remained current and preparations for the report our team needs to submit to CRA/CREU.

This report is due on May 5th.  We can adapt much of the content of the technical paper we created for the Swarmathon competition.  We will make necessary additions and revisions to include the requested sections on Future Work, website links, and Conference participation.  The efforts of our Outreach team will be described in this report as well.  Brittany and Ashlee have been asked to contribute to our Related Work section, in particular, since they were tasked with much of our literature review.

Catherine and I have been looking at various AV carts we hope to convince the Department to purchase so we can safely and efficiently take the rovers, router, and laptops to various outdoor sites on campus for physical testing.  Right now we have to remain close enough to a building to use an outdoor electrical outlet with a heavy-duty dual-outlet extension cord but we will explore the possibility of purchasing an inverter so we can get away from the buildings and the sensor noise generated by them.

For the remainder of the semester we will continue to address the previously identified goals.

Robotics short term goal:  1) continue testing code to address problems that persisted during the competition, 2) finalize means of publishing queues, 3) work on E&E.

Robotics long term goal: Improve upon current code to serve as a resource for the next team with emphasis on improving localization and implementing the Explore-and-Exploit algorithms.

April 20, 2017 – Swarmathon Competition


Three of our four Coders, two Outreach members, and a potential future Swarmathon team member attended the Swarmathon Competition at Kennedy Space Center this week so no formal research meeting was held.

News from the attending team members:

  • Some of our concerns about using older rovers for testing may be alleviated with the news that BCLab-UNM can provide replacement parts.  I think we will request new wheels at the very least.
  • We still had issues with one rover spinning endlessly.  We need to introduce a mechanism for determining if the rover has cycled through a full 2π and direct it to move to a new location if it exceeds a full-circle spin.
  • A rover became trapped inside Center – this happened to other teams in the competition as well.  Unclear if sun glare was an issue but we need to pursue this because we cannot afford to have a trapped rover pushing out returned cubes as it spins because exiting is thwarted by the presence of home AprilTags.  It should never ENTER center unless returning a cube, and then it should only cross the tags long enough to drop and then reverse.
  • Our High School team performed well on the random distribution of cubes, but did not do well for clustering.   For next year we will have the team develop better tools for each robot recording a cluster location so it can return to it.
  • NASA Internship opportunities quoted from Catherine Spooner’s blog:

They currently have two internship types.  NASA Intern, Fellowship and Scholars (NIFS) is the temporary internship that  last for about 10 weeks during the summer months or 16 weeks during the fall/spring semesters. The other internship type is called Pathways, and can lead to permanent employment.  The Pathways internships are also split into two categories: not-to-exceed (NTE) year long appointments, and year long appointments without a NTE cap.   After successful completion of a non NTE appointment, an applicant can be eligible to convert to full time employment with NASA.  For any of the internships, the applicants must be-a US citizen, have at least a 3.0 GPA and be attending an accredited institution.  NASA provides a website at https://intern.nasa.gov to apply for the NIFS internship.  To apply for the Pathways internship, you would go to https://www.usajobs.gov.  There are 10 main NASA locations that you could potentially work at; all of  them post to their internship opportunities on those two sites.

We will resume meeting next week and our goals will be revised then.  For now..

Robotics short term goal:  1) narrow down causes of poor performance and start testing code to address it, 2) publish queues, 3) work on E&E.

Robotics long term goal: Improve upon current code to serve as a resource for the next team with emphasis on improving localization and implementing the Explore-and-Exploit algorithms.

April 13, 2017

The meeting today was brief.  We reviewed a few logistical matters for those traveling to Kennedy Space Center for the Swarmathon Competition and then reviewed our current and future goals.

For now our changes to Explore-and-Exploit (E&E) will be tested in simulation.  We will be doing a better job of collecting data for later analysis, as thus far we determined most of our analysis was qualitative and not quantitative.   We need firm numbers to test how reliable and effective each variation of code is over a test period rather than depending on perceptions and the varied descriptions of each tester.

Some team members have revised their blogs as requested by the project lead to better meet CRA/CREU guidelines.  Everyone has felt somewhat overwhelmed by various deadlines and assignments but we agreed we need to meet our responsibilities in a more consistent manner, project lead included.

Robotics short term goals:  1) Attend the Swarmathon competition April 18-20 and determine how well our code worked.  2) Resume testing with detailed documentation of E&E code features and sim results.  3) Continue to build database so we can compare mapAverage values with expected values.

Robotics long term goal:  Improve upon the submitted code to serve as a resource for the next team.


April 8, 2017

The team met April 6 to review current goals and plan new ones.   As anticipated, our issues with Localization are a priority.  Whatever we can learn or strategies we can devise will aid the new Swarmathon team.  The code involved in fine and coarse maneuvering can be confusing – due to the lack of GPS precision, fine movements are handled with odometry and one new goal is to develop a better means of developing and sharing a combined grid.  As previously stated, each rover initializes with itself at the center of a x-y plane  ( the camera does use the z-plane for AprilTag manipulation, however ).   Once a rover actually returns a cube, it is able to reset its Center location to 0,0, which will improve precision for that particular rover.  Map positions are averaged, so we will pursue testing the error present in such averaging and see if it can be improved or offset in some manner.

The NetLogo code prepared by our High School teams was submitted but not without difficulty.  Can refer to personal blog by daphne for details on this.  In the future, our Outreach team will secure at least two contact methods for each student-leader as well as the Instructors, and provide a time-line of check-ins to confirm progress is being made as expected.

Short term Robotics goals have been met:   1)  Team completed and submitted the Technical paper by the deadline of April 4, 2359 eastern.  2) Team submitted final code on March 31.

Robotics short term goal added: Build database and compare mapAverage values with expected grid locations to determine if a margin of error can be calculated and used to improve localization.

Robotics long term goals are continued: 1) Attend the Swarmathon competition April 18-20 and determine how well our code worked.  2) Improve upon the submitted code to serve as a resource for the next team.

Outreach short term goals were met: 1) Attended the  workshop on April 6th and secured the final code from the students on the 7th.  2) Submitted HS code to github on April 7.



April 5, 2017

The Swarmathon Technical paper was uploaded to github last night and our code was submitted on Friday night.  We are glad to see those deadlines behind us.  More code improvements await, however, as well as the results of the actual competition. We would like to continue improvements on localization.  We would also like to revise our submitted code to include the timing triggers for Explore-and-Exploit.

Explore-and-Exploit is a dual-behavior algorithm that is guided by the time remaining in the current round.  The length of the round is published at the beginning of the round so the rovers can be assigned numbers and know the size of the arena.  For the first 60% of the round, rovers will utilize our spiral search algorithm to find and return cubes.  If clusters are found, the location and estimated number of cubes seen will be recorded in a queue.   Rovers will return to clusters they have recorded.

With 40% of the time remaining, rovers will convert to the Exploit Algorithm.  Rovers that have populated queues will publish those queues.  Rovers that are still searching will instead turn their focus to clearing the clusters found by their fellow rovers.

Introducing the timing trigger will be a new task for us to accomplish.


March 30, 2017

The team met today for testing and a group picture – not everyone was able to attend.   Casey from the core team was delayed so we will have to use an insert for him.

We worked on additional testing of the search algorithms we will be submitting.  At this point, we are still unclear if some testing results reflect rover handicaps or actual problems within the code.  It is especially frustrating to see different rovers continue to display such disparate behavior when running the code.  We still get the occassional rover who elects to just spin while the other does all the work, and does so effectively.  If it were the same rover each time we could probably pinpoint a physical or electronic issue, but it is not.

Short term Robotics goals:  1)  Team will complete and submit the Technical paper by the deadline of April 4, 2359 eastern.  2) Submit final code by 2359 tomorrow night.  3) Meet with Dr Bhattacharya on the 6th at 1400 to continue discussions and testing of  localization.

Robotics long term goals: 1) Attend the Swarmathon competition and determine how well our code worked.  2) Improve upon the submitted code

Outreach short term goals: 1) Attend workshop on April 6th and secure the final code from the students.  2) Submit HS code to github by April 7

Outreach goal met: Paper submission by the deadline of March 28.



March 23, 2017

The meeting today addressed our ongoing changes to search and the challenges encountered.  Dr Bhattacharya also outlined some tasks as deadlines approach.  I will lead the effort to submit our Swarmathon Technical paper on March 28th.  Xavier Graham will lead the effort to submit the Outreach Paper by the same date.

Dr Bhattacharya asked questions about how the rovers communicate, how they are uniquely identified during communications, and the role of GPS/odom/EKF in refining localization.

Localization has been an ongoing issue (even from last semester!) and on the surface it seems quite simple. But the code addressing it can be eyebleedingly complicated to read and understand.   I think if we were not pressured by deadlines and classwork, we would have written something new entirely.

Each rover starts considering itself at the center of an xy grid.  For the competition, they are arranged somewhere ‘roughly’ 50 cm outside Home base.  They may all be on one side of Home, or spread out around it.  No guarantees on disposition.  Thus, trying to create a shared grid is problematic and compounded by noise, imprecise GPS readings, and the degradation over time/distance of the wheel encoders.

Additional questions have been posed on the forums regarding this.

We are also working to improve a return-to-cluster functionality.  As I am sure every other swarmathon team is doing.

The HS teams meet at 4pm and the Outreach team will have a chance to go over their code ideas.

OutReach Short term goal: – 1) Outreach team will attend the follow-up Workshop tonight and review code the students have created.  2)  Grant Xavier github access

OutReach Long term goals: 1 )  assist the HS team with development and submission of an effective NetLogo codeset by April 6.  2) use materials and knowledge gained to produce the Outreach documentation required by Swarmathon and contribute to the success of our physical team. 3) Create and submit the Outreach paper, to be overseen by Xavier.

Robotics short term goals:  Goal regarding being able to use the April tag library in determining number of cubes and grid location of cubes is ongoing.  Goal regarding improving search and target delivery is ongoing.  New short term goal:  Recalibrate IMUS and see if this impacts testing results.

Robotics long term goals:  1) Unchanged — Produce code that will effectively search for, detect, identify, collect, and return target objects to a home base, aided by April Tags on the objects and at home base. 2) Added —  I was assigned to be lead on creating the Technical Pape that is due on Tuesday the 28th.  Other members to assist.


March 13, 2017

Spring Break!  Nonetheless, level 2 and Donny from Level 1 met today.  There is concern about the heading selected to return to home base after a rover has collected a target.  This heading is made using calculations made based on where it considers itself to be on the xy plane. If the rover does not see April Tags by the time it considers itself at/near (0,0), it begins to search, assuming it to be close-by.

For some rovers the initial heading toward home is made in the wrong direction.  In other cases, the rover is not close to home but begins to perform a search before even selecting a heading.  As this is not a consistent occurrence, it is difficult to pin down the exact circumstances that will elicit this behavior. We pored over code and discussed various options that we will test in sims and real-world for those taking rovers home from the meeting.

Dr Bhattacharya joined us at 2 via adobeconnect and we had to report our less than stellar progress.  We will continue to study the appropriate sections of code and make additional changes.  Additional questions posted to Swarmathon QA as well.

New lid supports have been printed. I did request a 3cm addition to their length.  As it seems the struts we printed during the summer – using a different material – have been breaking, I will request another set be printed so every rover can have a full set.

Goals remain the same for now.  In addition, Dr Bhattacharya asked me to submit some recordings of sims that illustrate the spin-search issue.

March 9, 2017

We met today with Dr Bhattacharya.

OutReach members worked on NetLogo.  Most had already completed to module 3 and finished Module 4 in preparation for our Workshop this evening.

Level 2 members spent a great deal of time discussing with Dr Bhattacharya issues with heading selection.  When returning a target, the rovers should calculate a heading that will return them to the center based on their current location. Some of the heading choices we have seen have instead pointed in the opposite direction of home.

OutReach Short term goal: – participate in Workshop tonight with the HS team members.

OutReach Long term goals: unchanged. 1) assist the HS team with development and submission of an effective NetLogo codeset.   2) use materials and knowledge gained to produce the Outreach documentation required by Swarmathon and contribute to the success of our physical team.

Robotics short term goals:  Goal regarding the pose question is in progress but demoted in priority.  Goal regarding searching and target delivery is continued since changes and testing has not resolved the problems identified. Question regarding this has been posted on the Swarmathon QA forum for review.

Robotics long term goal: Unchanged. Produce code that will effectively search for, detect, identify, collect, and return target objects to a  home base, aided by April Tags on the objects and at home base.


March 2, 2017

Parts of the meeting held today were conducted in focus groups.

Brittany shared her findings from the assigned research papers.  Dr Bhattacharya thought the virtual pheromone concept held some merit for our project.  In this type of search, some robots serve as ‘beacons’ and spread from home and assume stationary positions in order to receive and transmit data.  Other robots are walkers and spread from home and employ a random walk in search of their target.  When a walker finds a target -such as a food source in the insect world – it provides this information to beacons it encounters on its way home.

Information is shared between robots in regard to proximity to home and an identified food source.  In the example shown, a beacon at home base would have a home pheromone value of 100 while the one closest to the target – the food, in this example – would have a 100 level target pheromone level and a lower home value.  The beacons along a fairly direct path between home and target would have high values for both along a gradient.  More distant beacons would have significantly lower values, and over time all values would decay.

While none of our robots will be stationary, finding an effective method of communicating what they have encountered during their searches with the other robots could allow us to redirect robots that are experiencing a fruitless search while a comrade has found a stockpile of targets.


Casey describing how rovers return to base

The group then reviewed some specifics on the behavior of our rovers. Several pages of code were reviewed and changes were discussed that may improve our ability to control the movement of the rover as well as filter out noise encountered by the sonar.  More outdoor testing will take place so we can limit interference from metal, other wireless transmissions, echo, etc.  We also found some code we may think be in error.  Altering and testing will be easily determine if this is the case.

We are still working on the pose issue using the April Tag library but we hope this is something we can complete and utilize.

Spring break is coming up and we are exploring some dates we can meet during that time, possibly on the 13th with Dr Bhattacharya and on other dates with just interns.

OutReach Short term goal:  Meet next week to discuss any issues or concerns we encounter with the NetLogo modules.

OutReach Long term goal: 1) assist the HS team with development and submission of an effective NetLogo codeset.   2) use materials and knowledge gained to produce the Outreach documentation required by Swarmathon and contribute to the success of our physical team.

Robotics short term goals:  Previous goal to print home base met, thank you Catherine!   Goal regarding the positioning question is still in progress. New short term goal: 1)  make the requested changes to Search. 2) determine if the suspected error is, in fact, an error.

Robotics long term goal: Unchanged.


February 23, 2017

Today the group brought a rover in for some testing.   There are still behaviors observed that we cannot  reconcile with the current code as written.   It reverses when it should instead move forward to collect a target.  It runs over cubes while holding one.  And it still turns too quickly, losing potential targets in camera blur.

The previous short term goal of examining the stop-and-go code to explore possible changes to turning speed will need to be continued.  There is another Swarmathon webinar tomorrow and we will pose some questions about adjusting turning velocity.

Dr Bhattacharya has encouraged us to revisit something we had discussed earlier – using the April tag library to test to see if two tags are perpendicular – and thus probably on the same cube – versus are simply in close proximity.

Short term goals:  1) print out home base and include a return-to-home test after a target has been collected.   2) continue to look at possible ways to alter turning speed to improve target detection. 3) review Multi–Robot Foraging for Swarms of Simple Robots, a dissertation by Nicholas Roosevelt Hoff III as directed by Dr Bhattacharya. 4) See how we can use the April tag library to calculate positions of cubes in view relative to the camera.

Update Feb 24:   The team attended the webinar and Casey, as tech lead, posed questions to the UNM-Swarmathon panel about why angular velocity is used in some functions rather than simple velocity or speed.  (Angular velocity is the angle by which an object turns in a certain time frame and we reference that angle using radians.) and how we could have the rover turn more slowly when changing directions.  The panel directed us to some stored constants that can be manipulated to see if it elicts the response we are looking for.  Will update on the effectiveness of doing so.


February 16, 2017

Today we reviewed some rover testing results and images collected during testing, as well as went over some software issues.

The rover is able to locate and collect targets, but does not seem to detect potential targets while moving/turning as well as we suspect it could.    We located the section of code which allows the rover to select a new location on a random heading and move there.  Simple speed is not included directly in this function.   We will need to make some code changes and continue testing to see if we can make our turns efficient enough not to waste time, but slow enough that potential targets can be detected.

Short term goals: 1) work on start-stop code as above.  2) tech lead will delegate coding tasks as he deems needful.  3) continue to explore and test alternatives to random-walk.


February 9, 2017

I am in California at the AAAI-17 Conference so could not attend the meeting today.    From updates I have received, Brittany met with Dr Bhattacharya and will be reviewing papers that may assist us in developing Swarmathon code.  Her first assignment is Coverage for Robotics: A Survey of Recent Results1 by Howie Choset.  The others continue to on our current goal of reviewing and understanding the purpose of the Swarmathon code/functions.

1Choset, H.  Coverage for robotics – A survey of recent results.  Carnegie Mellon University, Pittsburgh, PA 15213, USA


February 2, 2017

Since we have some new members, our goal for this week is again to return to the Swarmathon code and review all modules.  Each member needs to have at least a general understanding of what each function does and how they are connected.

A longer-term goal (aim for mid Feb) is to decide how to replace the default random-walk algorithm with a more efficient search algorithm.  We are learning about informed and uninformed search algorithms in our Artificial Intelligence class and this has prompted discussion about the viability of various methods in the real-world and, specifically, in the competition environment Swarmathon employs.

Our long term goal remains to produce code that will effectively search for, detect, identify, collect, and return target objects to a home base, aided by April Tags on the objects and at home base.  Ideally, the code will be good enough to win the Swarmathon competition in April.

Since our project involves a competition, some of our goals may be posted in a generalized form, and our methodolgy may seem nonspecific. We hope to meet CREU guidelines without compromising our team’s contributions to new code.


January 26, 2017

See my personal blog for notes on the Seminar we were asked to attend today on Behavioral Cybersecurity: Studying Hacker Motivation, Personality, and other Human Factors.

Our team meeting addressed guidelines CREU has stressed they wish for us to follow in our documentation.

Our short-term goals this week:  1) updating our blogs to reflect what we are working on and what progress we are making toward our goals.   2) start a thorough review of the base Swarmathon code. 3)  Incorporate the searching algorithms we will be covering in our AI course into this project and see if the strategies described in text and lecture can improve the base code.

Our long term goal: produce code that will effectively search for, detect, identify, collect, and return April-tag covered 2x2cm cube to a home base composed of April-tags printed on a flat surface.


January 19, 2017

The research/internship group resumed weekly meetings today.  Dr Bhattacharya has offered his Artificial Intelligence class the option of joining the Swarmathon team as a means of replacing a couple of required projects, so perhaps we will see our numbers expand soon.

Everyone took turns updating the group on our individual status in terms of conferences registered for and summer internships we have applied for.

We ended the meeting early since Dr B and Dr C are assisting in the vetting of a potential new instructor.  There is a Swarmathon webinar at 1500 that we need to watch.  We hope it contains new information about the conditions of the competition, which is rapidly approaching.



December 12, 2016

I have reminded the CREU recipients in the Robotics group to be sure they have submitted their mid-project reports.  I had saved most of my responses in a text file, luckily, because Dr Bhattacharya has indicated he would like a copy of our reports for his records.

I think perhaps we were conservative on the average number of hours we spent on the project each week.   I guessed at an average of six, but it is probably closer to twice that if I include pure reading time.  I probably read an hour or so every night in bed — gotta love my little Samsung Note and Calibre Library for helping me convert pdfs to a more eReader-friendly format.



December 1, 2016 Meeting

Today was a working meeting for the most part.  All three rovers were brought to the meeting.  There had been issues with the communication between the laptop/PC interface and both Achilles and Aeneas crashing whenever it received keyboard input.  The GUI remains open and still relays information from the camera and sonar, but mobility communication halts.

Additional troubleshooting revealed that it was not actually the keyboard input that caused the crash, but the powering of the wheels in response.   A physical check of Aeneas determined one of his wheel connections was incorrectly inserted.  Once it was removed, flipped, and reinserted the crash resolved.  How it became flipped at all is unclear, especially since this error had not been evident in the past.

We also checked the crimping of the wires to the gripper, since its performance has been intermittent, but the crimpings in the pins appear clean.   There is some concern that the expansion board, marked in red below, does not seem very secure, but we cannot confirm this is the source of the problem.  Pressing down on the pins at the point of wrist connection (uppermost) sometimes serves to restore gripper function for a period of time.  I don’t think Swamathon will encourage percussive maintenance, however.

SEB Connected

In addition there was an alteration to localhost’s IP address in /host that had to be corrected on Ajax.   By the end of the meeting it appeared all rovers were functional, although we still have that “push here to make it work” issue with the grippers.

Josh has fixed his Ubuntu issues at this point.

Over the break we need to be sure we keep ROS updated since more frequent changes are being pushed out, continue to improve our understanding of the current code, determine where we can incorporate code we developed last semester, and add additional code we discussed but do not wish to advertise.


November 24, 2016 Meeting

Thanksgiving Holiday, no meeting today.  Continue on previously assigned tasks re the gripper and code development.


November 17, 2016 Meeting

We met in the Conference room today as our internship room was being used for a Forensics event.

Josh A, in the foreground, has been having Ubuntu issues on his laptop and continues to troubleshoot. Ubuntu has plagued most of us throughout this internship with sleepless nights and extra grey hairs because of missing dependencies or denied permissions (fairly easy things to fix) and odd occurrences such as booting to a blank screen at login, missing desktops (it was there yesterday, and now it’s gone?), or failing to boot at all.

A new member has joined the Joint Internship.  Samantha, seen here standing behind Catherine and Ryan, is getting an overview of each project and will decide which she wishes to focus on, or see if she has time and interest sufficient for both.  I suspect she already prefers computer vision / Surveillance.

Ryan spent some time with the Surveillance group discussing neural networks in image processing.  He then gave a presentation to the Robotics group about how data is processed in the Swarmathon code over ROS topics.  There is also a little tutorial online.  We discussed current Mobility and Obstacle Detection code and functionality, and methods we could employ to improve the movement and speed of the rover after a target is detected so we can approach and collect with a bit more finesse than the rover currently displays.  Indeed, Aeneas has a habit of rushing the target and running over it.  My theory is he plans to crush the cube and then eat it.

At any rate, Ryan helped us focus on several tasks we need to address in the coming weeks. He is graduating this semester and we will miss his expertise and input!  His internship experience in Computer Vision has gained him an excellent position at a firm in Virginia, however, so he is very happy.



November 10, 2016 Meeting

We had interruptions today from repeated fire alarms in the Science Tech building. Eventually we were able to remain inside for an abbreviated meeting.  Dr Bhattacharya discussed computer vision with the Surveillance team leader while the Robotics group set up a router and SSH’d into one of the rovers to demonstrate the gripper.  However, it was discovered some of the updated modules were no longer on the rover.  I am beginning to suspect we’re jinxed.

After some compile errors revealed the problem, the April Tag library / new modules were downloaded again and it compiled correctly.   We will arrange to meet early in the week to work on gripper testing, updating the other rovers, and deciding what tasks can be delegated so we can make progress on rover-target interaction.


November  3, 2016 Meeting

Today we focused on the SNCURCS conference and interfacing with the rovers status post the last update.

Most of us have opted to drive our own vehicles to the conference.  There is a bus transporting other presenters in the RISE program – of which Sam is a member – but they will not be able to accommodate the rest of us.

Drs Bhattacharya and Czejdo made sure everyone had their poster ready – or were picking it up by Friday.    Ryan did a dry-run of his presentation with pertinent questions posed and suggestions provided by Dr Bhattacharya.

John is leaving the internship due to time constraints, Catherine will assume the role of Project lead on the Human Tracking/Surveillance side.

In another part of the room, the Robotics team was confirming gripper functionality.   Hostnames were updated but SSHing into the rover only allowed viewing through the rover’s camera.  It did not permit gripper control, nor even motor control.    Troubleshooting did not yield any result by the end of the meeting.  Will follow-up on this after some research in Swarmathon/NASA databases.

Update:  Casey posted the issue on the NASA Forums and the problem was the host name associated with the laptop Josh was using.  It was given a variable name rather than the machine name of his device.  Glad it was not hardware related.  All seems to be working well at this point.

RISE: Fayetteville State University Research Initiative for Scientific Enhancement 


October 27, 2016 Meeting

I was absent today due to illness, but Brittany indicates much conversation focused on the materials to be included on our conference poster.  I’ll request team members submit any pertinent screenshots and write-ups of their contributions/progress and put everything together, then have them review it before I submit it to Dr Bhattacharya for his input.


October 20, 2016  Meeting

Dr Bhattacharya indicates the computers in room 335 will be updated to provide dual booting Windows/Ubuntu and that class room can be utilized by the internship as well as room 213 (which is a small office).

We discussed at some length the newly released April Tag Library and potential updated we can employ to improve target localization and rover positioning.  No specifics to be posted here!   Sorry competitors!


October 19, 2016

Our Robotics abstract has been approved for the SNCURCS conference!   Casey, Joshua, and I have registered to attend.  Donny will be with the National Guard that weekend.  I’ll get updates from Brittany and Ashlee, hopefully they can attend as well.

Casey updated two of the rovers today and I’ll get the third tomorrow or Friday.  My plan to do it Monday was sidetracked when an exam that the Hurricane postponed was scheduled for our first day back Monday, so I focused on that.

Donny’s adventures with Ubuntu continue and I think perhaps we should initiate research into this amazing phenomenon.  He purchased a Macbook, I believe – an iOS device, at any rate, and instead of a dual boot installation he replaced the iOS with Ubuntu.  He’s maintained good humor throughout.

As part of the updated Swarmathon code, we now have an April Tags library.  Dr Bhattacharya wants us to familiarize ourselves with this library because he is pondering whether we can shift focus from our object detector and instead use/tweak the functions of the library to estimate the 3D pose (position and orientation) of the cube. To quote him: “Here the end goal is to precisely map the cube (eighth vertex locations) in the 3D coordinate space of the robot from image, since that will allow robot to precisely orient to grip the cube.”    He has provided documents to assist in our understanding of this.


October 13, 2016

No meeting this week.  Indeed, the entire University is closed because of Hurricane Matthew and the severe flooding the area has sustained.  Prior to the Hurricane the area had experienced a great deal of rainfall, thus when the storm hit the ground was already saturated.  Simple run-off flooded many low-lying areas, while 10 damns in the area were overcome.  Many of us were without power for quite some time – four days for me, some still have no power or water.  So, the only work I’ve done this week is to clean out the freeze and fridge of spoiled food, gather up storm debris, and help my uncle cut down the 8 or so trees that fell in our yard.   Well, amend that:  the first thing I did when the power came on Tuesday was to submit our revised SNCURCS abstract.


October 6, 2016 Meeting

Dr Bhattacharya requested we address a few administrative items today in our meeting. Project leaders are instructed to submit a report describing our group’s project goals, our current progress toward these goals, and list the URLs of our blogs.    We submitted the requested materials to him via email.

Both groups are then tasked with reading the newest version of Learning OpenCV 3. (1)  It is felt the instructions in this document may prove more useful for the development of our object detector than the Blueprint document.  Using the revised Learning OpenCV3 will require a switch of our detector code from Python to C++.


1 Kaehler, Adrian and Bradski, Gary.  “Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library.”  O’Reilly Media, Inc., Sebastopol, CA.  Early release of First Edition, November 11, 2016.



September 29, 2016 Meeting

Dr B’s meeting tasks:

Abstract progress

Blog status

Milestones in project efforts

Challenges faced by team/team members

Deadline of abstracts stressed. Content was reviewed again.  Dr Bhattacharya requested to view some of our blogs, so we displayed this blog – project and personal, and several other members of the internship shared their sites as well.  Ongoing and timely updates are needed.   We split into groups and reviewed annotations and discussed problems we were facing.  Casey and Joshua continue to shine and serve as an example to the rest of the Robotics team. Approximately 800 images have been configured as positive and training images.  The detector is picking up some false positives – like an oil stain in the parking lot behind the image of a cube  – but oddly enough the alert on the cube itself is very small.  It is not marking the entire cube as a matched image, but only perhaps a fifth of it.  The match areas seem to be of a consistent size and Joshua indicated he’d made numerous resolution changes to the code.   Dr Czejdo recommended we take more images of single cubes with less clutter in the background and continue to train the detector, while also exploring a possible Uniform Density factor in our positive-match box size.  We spent much of the meeting time working on Donny’s Toshiba Satellite notebook, which has proven resistant to Ubuntu installation.  He states he may simply purchase a laptop.  Some members of our group stayed a bit late after the meeting to continue working on our abstract, trying to get it within the 300 word limit listed in the SNCURCS guidelines.  Brittany has volunteered to be primary on this so we can submit for approval prior to the submission deadline.


 September 22, 2016 Meeting

Today was a bit of an abbreviated working meeting.  Everyone reported on their progress.  Many are still having technical issues.  Dr Czejdo again discussed the SNCURCS on November 5.  The content and focus of our abstracts were discussed, and he stressed that it was understood that our eventual presentation poster would reflect far more information than the very short abstract could encompass at this time.  We shared ideas about what type of images and screenshots we could employ on the poster to show our progress, and even some of our pitfalls and obstacles.  We should state our goals, mention any current challenges and our methodology for addressing them in the abstract.

We also briefly reviewed some annotations results and decided we need to continue to gather additional samples for training purposes.


September 15, 2016 Meeting

Casey reports 328 images thus far have been annotated for the cubes.

Ryan asked us to be alert to the following:  213 PC:  2 senior project groups are using the computer remotely, so you may need to text Ryan first to see if it is in use.     Then boot into the first Ubuntu partition. When finished, reboot and choose the 2nd partition so the projects group can access it again.   The laptop in the room can be used freely.

Conference participation.  There will be a State of North Carolina Undergraduate Research and Creativity Symposium on November 5 in Durham on the campus of North Carolina Central University.   Deadline for abstract submission is October 3rd.  We need to submit the required documentation, then start working on posters.   Define our goals, challenges, methods.  This will be an excellent venture for everyone in the group, allowing us to learn from more seasoned presenters and researchers, extended our network, and gain valuable experience!

Catherine was moved to the Surveillance team.  We miss her already!

Ryan, Casey, and Josh worked on troubleshooting a memory-related issue with the minimum-dimension script.  Members are combining annotation files to generate a larger test bank.


September 8, 2016 Meeting


When annotating, get at least 75-80% of the image in the annotation box.    Ryan reiterated that everyone should clone the material from GitHub as described in the Blueprint eBook and that they’ll then have access to the described commands for annotation and image processing.

Then the annotation file is used to generate negatives and positive training images.   An n or p appears in the filename to indicate if it’s a negative or positive training image.

Joshua experienced issues installing OpenCV3 with 2.4, is working on resolving dependencies.

Assigned Tasks:

Find and distribute the video from 213, see if it can provide enough images. Sam Cooper

Method by which to extract still frames from video. Lee Gibson will explore this.

Annotate images per the instructions in the eBook. Open the source code to see what key commands you need to save, advance, etc.   Brittany and Casey will bring a file of annotated images for the next meeting for the Robotics Group.   For Surveillance, John will address this.  Groups need not wait for images from the video or swarmie cubes, can use images from the web to start learning how to use the annotation software while waiting for other, more specific, images.

Documents for CREU due on the 15th, Dr B would like them done today.

Blog – Dr B desires them to follow the same format. Everyone could use WordPress and follow a similar pattern as the blogs used by Cooper and Faircloth.     Everyone has been added to GroupMe for ease of communication, encouraged to look at current blogs and try to make similar ones.

Poster Presentation: Dr B said we could focus on the early tasks of image detection for posters, need to decide on number of posters per group, 1 or 2.  (1)


September 1, 2016 Meeting

Both groups met today at 2pm.  Our objectives were posted earlier in the week on Canvas and were as follows:

Finish anything that is still unfinished. Next, read the next chapter of the ROS book (for robotics team members) and OpenCV book (for surveillance project team members) and try to run code examples.

This week both teams – the robotics and surveillance, will work on object detection in image & video. The reference book is “OpenCV 3 Blueprints.”  (1) Chapter 5 of that book shows the step-by-step process of how to use OpenCV to create an object detector. Details follow.

Task description for the robotics team:  create a target (april tagged cuboid) object detector.

  • Collect positive & negative examples. Ryan has the cuboids for which many images should be acquired from varying poses. Using the Swarmie camera is preferred.
  • Create HAAR-like wavelet feature model
  • Create local binary pattern model
  • Use cross-validation to achieve the best results
  • Collect more positive & negative samples to improve above results

Task description for the surveillance team: create a person detector.

  • Collect positive & negative examples using the video data set collected from the Millennium project. The workstation in ST 213 has those file.
  • Create HAAR-like wavelet feature model
  • Create local binary pattern model
  • Use cross-validation to achieve the best results
  • Collect more positive & negative samples to improve above results

For our Robotics team, we started taking images during the summer of our cubes and various other items to build a positive and negative library.   We’ll work on adding to this library.

Some human surveillance video recorded in an earlier internship is on the 213 desktop and will be made available for the Surveillance group as a starting point.

The OpenCV 3 Blueprint e-text  was reviewed in detail during the meeting, especially in regard to the differences between object recognition, detection and categorization.   Dr Bhattacharaya emphasized that we are employing object detection and reviewed the steps to be taken:   Data collection, training, detection, and then validation using a new sets of images.  The section on image annotation was also reviewed.

Ryan Fisher will be setting up user accounts on the 213 desktop after partitioning the drive.  He was able to install Ubuntu on a laptop for one student and said it was a difficult process.  He’ll install OpenCV next.

A concern and question was raised about running OpenCV 2.4 ( required for ROS) alongside OpenCV3 (for our object detection and surveillance).  A few of us experienced issues running both during the summer, noting that ROS seemed to be impacted.   Perhaps we can resolve this by checking pathing and doing some testing.  If it proves problematic, we’ll simply use OpenCV3 for now  until our images have been secured, and then we can revert back to OpenCV2.4.

Two additional students joined us for the Internship, Josh and Casey.  Funding for them and our other non-CREU/CRA internship members may soon be obtained. More to follow on this.

The meeting ended with these tasks to be addressed.


1 Joseph Howse, Steven Puttemans, Quan Hua, Utkarsh Sinha.  “OpenCV 3 Blueprints.”  Packt Publishing Ltd, 2015.

August 25, 2016 Meeting

Ryan Fisher led today’s meeting .  Drs Bhattacharya and Czejdo had a faculty meeting and joined us toward the end of the hour.

Confirmation was requested that everyone had completed the assigned readings.  Participants in both groups were required to read the CRA/CREU proposals.  The Robotics group was to read Chapter One of “Programming Robots with ROS” (1)  and to confirm installations of Ubuntu 14.04 and ROS on their machines. The Surveillance group was to read Chapter One of “Learning OpenCV 3rd edition.” (2)   All materials were posted on Canvas, our University’s Learning Management System.

The creation of Piazza accounts was again stressed for those that had yet to make one, as well as blog creation. Various blog software was discussed, and Dr Bhattacharya encouraged students to communicate with CREU/CRA staff as needed to determine any formatting or content requirements.

Access to laptops was revisited, with additional departmental laptops becoming available for those in need.  The desktop in Room 213 of our Science and Technology building will also be available for student use.

The meeting concluded with a reminder that any incomplete tasks should be resolved as soon as possible, as new tasks will be posted on Canvas soon.

Some time was spent after the meeting troubleshooting installation issues for a few participants.


1 Morgan Quigley, Brian Gerkey  & William D. Smart. “Programming Robots with ROS: A Practical Introduction to the Robot Operating System.” O’Reilly Media, Inc., Sebastopol, CA.  December 2015.

2 Kaehler, Adrian and Bradski, Gary.  “Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library.”  O’Reilly Media, Inc., Sebastopol, CA.  September 2015..

Initial Internship Meeting

The 2016 Fall Semester started Tuesday and our Internship group had its first meeting on Thursday.

It was decided the Robotics group and the Surveillance group would both meet, for now, every Thursday at 1400.  Later, if scheduling conflicts can be alleviated, the Robotics group will meet separately.

We reviewed a few early objectives – laptop access, dual-booting Ubuntu, installing ROS and OpenCV, Piazza account creation, blog creation.

A campus tour of Penn State had been discussed as part of our exploration of potential graduate study, but the tour required that the GRE already be taken.  As this is not the case for  most of the group, other options will be explored.    I was interested in the one-year Master of Engineering in Computer Science program they had for those interested in working in industry rather than pursuing research positions.  Hopefully we’ll see such a program elsewhere where a tour is possible.

Our AGORA summer internship ended on June 29th and three of us from that group met at the University  before Fall classes started and had Achilles (one of our three Swarmathon rovers) start taking images of the cubes similar to those that will be employed at Swarmathon next year.  We did our best to vary angles, configurations, number of cubes per image, and shifting backgrounds, including white lines on handicapped parking spaces, oil leaks, even trash items of our target colors.  Using image annotation, we’ll have some positive and negative images to help our rovers ‘learn’ how to recognize the cubes so they can maneuver close enough to grasp them and return them to home base.

I have Ajax at my  home right now and have been working on tweaking localization using files Ryan provided.  We determined we were getting too much interference at the University, so those of us that live in rural areas may do a lot of real-world testing at our homes.

Looking forward to this CREU/CRA internship, hoping to learn a great deal and contribute something worthwhile.