BroncoSwarm – daphne’s blog

Personal internship progress and challenges

Reminder: project is part of a competition.  Information regarding code and methodolgy may therefore be described in very general terms since this site is necessarily public.


May 4, 2017

This week the team was tasked with completing the report we need to submit to CRA-CREU.   Earlier in the week I asked each member to review the guidelines and submission requirements and to submit personal materials, comments, summaries, literature reviews, and suggestions so we could combine them with material already submitted as part of our Swarmathon technical paper.

As items are submitted, I am incorporating them into the final report.

This evening Ms Spooner and I met with Dr Cjzedo to discuss a few things he felt we should include in the paper, such as future plans for our team members, not just future plans for this project.

Until May 14th, I will not be working on robotics and instead will focus on a protein modeling project for our AI class.  During the summer I will be learning Java and engage in a few AGORA sessions with high school students as part of a Center for Defense and Homeland Security internship.  I will also continue working on sims to address the problems we found persisting at Swarmathon.

Short term Robotics Goal: Submit paper by May 5th

Long term Robotics goal: Continues to improve upon current code to serve as a resource for the next team with emphasis on improving localization and implementing the Explore-and-Exploit algorithms.

April 27, 2017

Dr Bhattacharya discussed the final stages of the current internship today.   Our report to CRA/CREU is due on May 5th and we are adapting our Swarmathon technical paper as needed, as well as introducing new material to meet report guidelines and include a description of our Outreach program and our future goals there.

I am still running sims at home to test various code changes, but still get rovers exhibiting variable behavior in response to the same stimulus.   Physical testing is what we need to prioritize, as previously discussed,  because we cannot trust the veracity of what we see happening in simulation.  Catherine and I have been looking at materials to address the logistics of this plan.

As I help the others work on the report, I will continue to pursue our current goals, in sim for now since all the rovers are with other team members, but if I arrive at some promising code I will send it to one of the others for physical testing.

Robotics short term goal:  1) continue testing code to address problems that persisted during the competition, 2) finalize means of publishing queues, 3) work on E&E.

Robotics long term goal: Improve upon current code to serve as a resource for the next team with emphasis on improving localization and implementing the Explore-and-Exploit algorithms.

April 20, 2017

Most of the research team is in Florida at the Swarmathon competition this week, so we have not met as a group.

I have continued to work on sims this week, but given what I saw in video from the first rounds, I shifted from the E&E algorithm to trying to pinpoint why our rovers did not perform well in preliminaries.    The team members present said they were not allowed to observe the simulation environment running on the laptop, so they could not determine what sensor data the rovers were responding to when they failed to behave in an expected manner.

We, as well as other groups, experienced a rover passing into Center, which is something we had occur in simulations and physical testing before the competition and had attributed to communication delays or overtaxed processors.  That it still happened with competition-ready rovers and to other groups as well made me wonder if the Center placard was reflecting sun glare and the camera was not registering the home AprilTags. Since the group was not able to observe sensor/camera information, this remains speculation. The team did say the Center placards were warping due to the heat.

One rover was caught in a spin loop while searching.   We had hoped this, too, was a simulation issue but clearly we need to introduce a failsafe that if the rover has exceeded a 2π revolution then it should select a new location on a new heading and move there before resuming a search pattern.

We must focus more on physical testing.  I do not know of any indoor arenas we can use on campus, so we may have to simply use the area outside our Science and Technical building so we can have access to power.  A location further away from metal would be ideal, however, should power be available.

Robotics short term goal:  1) narrow down causes of poor performance and start testing code to address it, 2) publish queues, 3) work on E&E.

Robotics long term goal: Improve upon current code to serve as a resource for the next team with emphasis on improving localization and implementing the Explore-and-Exploit algorithms.

April 14, 2017

Four members of our team met yesterday; Dr Bhattacharya had faculty obligations that kept him away for but a brief moment at the opening.   Most of that discussion concerned the planned trip to Kennedy Space Center and making sure the students that are going are suitably prepared.

I have been running sims using the search algorithm the team submitted.  I am hopeful my laptop will be returned from repair soon, as the departmental laptop I have been using is limited in the number of rovers it can sim before becoming unstable.  My laptop required a BIOS reset after a freeze zipping some simulation videos, a confusing result but, this internship has proven over and over that no laptop is safe!

I am introducing an elapsed-time monitor so the search behavior of the rovers can switch from the standard search to the “grab what we know is there because time is running out” exploit algorithm.   For the exploit portion, I anticipate I will need to confer with Casey and Josh about the best way to have each rover publish their queues – which contain cluster information –   so they can be accessed as globals.  For now I am reviewing the Robot Operating System (ROS) to improve my understanding of the publisher and subscriber system of communication between rovers because I would like to write that code myself.  ROS is an area I think we all neglected.

I received notice after the internship meeting that my maternal aunt died.  My mother lives with me so I will be juggling a few things this weekend dealing with familial obligations.

Robotics short term goal:  1) work on E&E in sims, address the publishing question regarding the queues.  2)  team members will attend the Swarmathon competition April 18-20.  As stated before, I opted not to attend because I do not wish to miss class, and would have had to cancel anyway due to the death affecting our household.

Robotics long term goal: Improve upon the submitted code to serve as a resource for the next team with emphasis on localization and the Explore-and-Exploit algorithms.  This will be my focus for a while.


April 8, 2017

At the Outreach workshop on the 6th, the students seemed very close to a merger of their code.  Terry Sanford had code that searched from center in a sweep that was effective but not efficient.  Jack Britt had code that created six different robots with varying behavior.  Some worked in a square-search pattern, with one working outward and another moving to the edge and then working inward.  Others did sweeps or followed pheromone trails left by robots dedicated to searching and collected what had been found.

The teams agreed the two least efficient JB robots would be replaced by a variation of the sweep code written by TS.

At noon Friday revisions had been made to the code in the repositories of each school, however, not as expected.  The efficiency of the TS code had been improved, but the header did not include names of all teams members nor the Instructor mentors, so it did not look like a merged effort. In addition, it had been written in the wrong version of NegLogo.   The JB code was now lacking two robots but did include JB team and Instructor information in the headers.  It also did not represent a merged effort.  I sent an email to the Instructors asking for an update.  Shortly after 4pm I sent another one, having heard nothing.

Later that evening Dr Bhattacharya was able to reach one of the students from Jack Britt and that young man sent me the six-robot code they had worked on.  They had mailed a four robot code to the TS student leader, but it is not clear why the expected merger never took place.  In email exchanged with TS after a teacher reached the student leader, the needed changes to the header of the TS code were explained but were never made to the repository.  Dr Bhattacharya decided since the Jack Britt code met the submission guidelines and passed the provided tester, we would submit that code to github.

In the future, we need to make sure we have contact information for the student team leaders and that we establish check-in times where they update us on their progress and we confirm they are meeting guidelines and will meet deadlines.  We did not want to have to submit code that did not represent the work of both teams, but little choice was left.

Short term Robotics goals have been met:   1)  Team completed and submitted the Technical paper by the deadline of April 4, 2359 eastern.  2) Team submitted final code on March 31.

Robotics short term goal added:  Build database and compare mapAverage values with expected grid locations to determine if a margin of error can be calculated and used to improve localization.

Robotics long term goals are continued: 1) 3 core team and 2 Outreach team members will attend the Swarmathon competition April 18-20 and we hope our code performs well. 2) Improve upon the submitted code to serve as a resource for the next team with emphasis on localization and the Explore-and-Exploit algorithms.

Outreach short term goals were met: 1) I participated in the workshop on April 6th and secured the final code from the students on the 7th and,  2) I submitted HS code to github that evening.


April 5, 2017

I submitted the Swarmathon technical  paper for our team last evening.  The team voiced they were satisified with it and we are all glad to have the code and paper submission behind us.  Our code was submitted on Friday.

We will now continue to look at our code and implement the changes we had wanted to, but did not have time to because we were beset by localization issues.  Now we can fine tune localization until we can get consistent test results and then start adding more robust and innovative alogrithms.

Team meeting tomorrow.


March 30, 2017

Huge sigh of relief on Tuesday the 28th when NASA Swarmathon officials agreed to the deadlinestress_co111requests from multiple schools to change the deadline of the Technical paper to Tuesday, April 4th.   Our team was trying to address continued issues with localization and make changes to the search algorithms we settled upon, so trying to write a paper before we had even submitted a final code and reviewed final test results was problematic.

They did not agree, however, to extend the deadline for the Outreach paper, so I assisted Xavier in its creation and then submitted it to github.  The same issue existed there — the High School students have not submitted final code and completed final testing, so much of the results discussion involves qualifiers such as ‘to this point’ and ‘anticipated.’

Our final code will be submitted tomorrow evening before midnight.  The team met today to continue testing of our search algorithms.  Localization issues still generate some frustrating but occassionally amusing aberrancies in rover behavior.

Outreach goals: 1) Goal regarding Outreach paper submission is met.  2) Outreach team will meet with the HS teams on April 6th at 6pm to review code.  3) Push the the HS code to the appropriate branch by the deadline of April 7.

Short term Robotics goals:  1)  Finishing writing the paper and submit it by April 4 at 2359.  This task has been assigned to me and I have written most of the paper but need additional data that will only come from final testing.   2) Meet with Dr Bhattacharya on the 6th at 2pm to continue discussions and testing of  localization.

Robotics long term goal: 1) Produce code that will effectively search for, detect, identify, collect, and return target objects to a home base, aided by April Tags on the objects and at home base. 2) Attend Swarmathon.  I will not be going, I do not wish to fall behind in class but the rest of Level 2 and Donny from Level 1 are going.    3) Improve upon this code.



March 23, 2017

Last week I ran sims and recorded a few videos of some odd behavior from rovers in response to what should have been equivalent stimuli – and my laptop froze while trying to zip the files prior to emailing.

Eventually I had to do a hard restart and afterward the laptop would not boot at all.  I typically dual boot using an external drive as my Ubuntu drive.  However, it seems the bootloader was not working at all, I was offered no option to select Ubuntu or Windows, and I could not enter BIOS.

Connecting the external to my desktop introduced a new world of pain since it was not even recognized as a drive by File Manager or Disk Management.  After trying each usb port in turn, I eventually found one that recognized it in Disk Management and I was able to assign a drive ID and then use my disk utilities to scan and repair.

Launching into Ubuntu allowed me to access the Ubunutu partition and retrieve my files – one of which was a java file I needed to turn in as a Project assignment for another class.  I usually transfer files I have been working on to a flash drive dedicated to classwsork before leaving Ubuntu.  That day, of course, I did not get a chance.  My relief at seeing those files!  I  had performed a backup about six days prior so only the vids and the java file were the real concern.

For now I am using the spare swarmathon laptop from Room 213 and will need to send mine in for a BIOS reset.  This internship has been rough on laptops!

At the meeting today we focused again on the searching algorithm and issues we continue to have regarding localization.   We have been communicating on the forum regarding the interaction of GPS, odom, and EKF, as well as exploring ways we can improve how the rovers communicate with one another.   Some of the strange behavior we have seen has prompted a suggestion from the forum leaders that we should recalibrate the IMUS of our rovers.

Since the deadine is approaching I was told to focus on Level 2 assignments for now. Another team member will work on the Outreach Paper and I will work on the Technical Paper due next Tuesday.  I have already asked everyone to go through their notes, screenshots, pictures, and send me what they see as representative or as milestones.

Outreach goal: Add Xavier as an authorized user for our repository if he has a github account.  If not, we can push the the HS code to the appropriate branch by the deadline.  I have created the reports folder with a placeholder readme file.

My robotics short term goal this week is get the paper written!  I will not be doing any additional sims for now – the cloning to the departmental laptop did not go well, so I would need to spend time I do not have troubleshooting that even if I wanted to run sims…

Robotics long term goal: 1) Unchanged. Produce code that will effectively search for, detect, identify, collect, and return target objects to a home base, aided by April Tags on the objects and at home base. 2) Submit Technical paper by the deadline.

March 13, 2017   Spring Break

The Swarmathon level 2 group and Donny from Level 1 met for a bit over four hours today to go over testing, code revisions, challenges.  We have been reviewing the functions controlling rover mobility, searching and target delivery.  It seems like with every test some new and unexpected behavior is noted.  We reported to Dr Bhattacharya over adobeconnect at 2pm.

We are addressing some issues with mapping, as there is concern about direction choices made when a rover has collected a target and should then select a heading toward home base – using calculations made based on where it considers itself to be on the xy plane. If it does not encounter home base on that heading when it thinks it has reached 0,0, then it begins a spin-type search.  For some rovers, the initial heading is not made, or is made in the wrong direction.  As this is not a consistent occurrence, it is difficult to pin down the exact circumstances that will elicit this faulty behavior.

Dr Bhattacharya asked me to submit some recordings of sims that illustrate the spin-search issue.  The three sims I have run this evening, of course, have not replicated the odd behavior I saw yesterday.  The same thing happens when I hear an odd noise and take my Jeep in for service. Mechanic says “cannot reproduce reported problem!”  I will run more and see if I can capture it.

Perhaps we can learn something from NetLogo here.  There we can just ask it to face (0, 0) and move forward!  I need to discover if I can find the ‘under-the-hood’ code that makes that simple command work and compare it to the calculations in Swarm code.

Goals continue as previously written, plus I will continue recording sims and send to Dr Bhattacharya ones that illustrate our concerns.

March 9, 2017

We again broke into groups today at the regular meeting.  I and the others involved in the OutReach portion continued working on NetLogo modules.   Most of us have finished at least module 3 and I had completed Module 4 barring an issue with an If-else loop. Donny had experienced the same issue and pointed out that I was missing part of the following section of code.  I’m still not clear why it would not compile as the two sections were independent of one another, as far as I can tell, but I will revisit it.

Previous OutReach Short term goal:  complete all modules and be prepared to participate in the workshop scheduled on the 9th with the High School teams — Met.

OutReach Short term goal: – participate in Workshop tonight with the HS team members.

OutReach Long term goals: unchanged. 1) assist the HS team with development and submission of an effective NetLogo codeset.   2) use materials and knowledge gained to produce the Outreach documentation required by Swarmathon and contribute to the success of our physical team.

Robotics short term goals:  Goal regarding the pose question is in progress but demoted in priority.  Goal regarding searching and target delivery is continued, as our testing has not not been conclusive and new inconsistencies have been noted.  We did make a small correction to the base code that we suspect was placed there deliberately to see if it were discovered.

Robotics long term goal: Unchanged. Produce code that will effectively search for, detect, identify, collect, and return target objects to a  home base, aided by April Tags on the objects and at home base.

** Update OutReach: Met goal regarding Workshop participation.  7 of the 8-member Jack Britt High School team joined us, as well as three from Terry Sanford High School.  Dr Bhattacharya presented an introduction to NetLogo via PowerPoint and then took questions from students.

Commands and concepts were clarified, such as ‘let’ vs ‘set’ and the purpose of ‘ask.’  Most of the students had finished Module 2 and were working on Module 3.   One student from Terry Sanford was working on Module 5, the competition code.

Members of our internship checked with individual students on their progress and any questions or obstacles they had encountered.  I was able to assist one student identify an issue with brackets that was preventing his code from compiling, but no serious  problems were found.  The modules are well-written and clear and the HS students were able to follow them easily.

I provided the students with my email so they could request to join an optional GroupMe channel to discuss NetLogo issues and ideas.  We will meet again on March 23rd.

OutReach Short term goals: – 1) participate in follow-up Workshop and 2) assist students with any emailed or chat questions posted.

OutReach Long term goals: unchanged.

March 7, 2017

Four members of our OutReach group met today to complete more of the NetLogo modules. An issue I had with ‘shape’ was resolved when I found a mismatched set of brackets!   I have to remember to turn off my touchpad, as this has been a frequent source of inadvertent text highlighting and subsequent deletion.  It cannot simply be that I failed at typing.   None of the others have encountered any problems.

Thus far the modules seem straightforward and well designed with increasing levels of complexity.  I do not think the High School students will have any difficulty implementing them and I think they will enjoy the immediate feedback they receive as they add code then run the program to see how it changes the behavior of the simulated robots.  I am curious to see if we learn something from this we might find useful in our own competition.

OutReach Short term goal:  complete all modules and be prepared to participate in the workshop scheduled on the 9th with the High School teams.

Previous OutReach Short term goal met – meeting was held.

OutReach Long term goals: unchanged.


March 2, 2017

Meeting held today, with some of it conducted in small focus groups.

Brittany summarized assigned research papers, including “Multi-Robot Foraging for Swarms of Simple Robots” cited on the Project page. She described the three algorithms for search and coverage explored in the paper, including virtual pheromone, gradient, and sweeper.

Dr Bhattacharya feels the virtual pheromone concept (discussed more on the Project page) may provide some good ideas for our project.  The closest analogy to VP I could find was the children’s game of Hot-or-Cold Hide and Seek.  The walkers update beacons on target and home proximity, and other walkers then retrieve that data and upload their own.  In this way, a beacon call tell a walker if he is getting colder or warmer in reference to home or target, depending on the pheromone being evaluated.

We may can use this to improve how our rovers communicate to one another where a target-rich area is, if we can develop a means to reconcile their individual grids with a larger whole.

As a group we reviewed recent testing and discussed various sections of the code that guided behavior while searching and behavior once a target had been retrieved.  Indoor testing may itself be the source of odd rover behavior, since the sonar could be responding to echo and there is always some interference near metal.  Our focus will shift to outdoor testing.

Some code changes were suggested and will be implemented and testing.  As noted before, details on code will not be discussed in this blog due to the competition.

As noted in the project blog, we are still working on the pose issue using the April Tag library but we hope this is something we can complete and utilize.

OutReach Short term goal:  Meet next week to discuss any issues or concerns we encounter with the NetLogo modules.

OutReach Long term goal: 1) assist the HS team with development and submission of an effective NetLogo codeset.   2) use materials and knowledge gained to produce the Outreach documentation required by Swarmathon and contribute to the success of our physical team.

Robotics short term goals:  Previous goal to print home base met.   Goal regarding the pose question is still in progress. New short term goal: 1) make changes to the code as discussed and record results.

Robotics long term goal: Unchanged.


February 23, 2017

Today we were testing the rover as it implemented a search for targets.  It was able to collect a cube as in the past, but we still see a few bugs.  For example, while searching it came close to a cube and was turning – to me it seemed as if it detected the cube at the limit of its camera vision and turned back to approach it – but instead of going forward it reversed, selected a new heading, and reversed again.  In another test it repeated a behavior we have seen before – running over cubes in its path.  This time it was holding a cube and either could not detect the cube in its path, or did not respond to it using collision avoidance as it should have.  Changes made to see if alterations in speed would improve target detection have not produced measurable outcomes thus far.

However, an issue also exists when there are multiple cubes in a cluster – how can the rover differentiate between them and select one to collect.  So we have identified short term goals as: 1)  use the April tag library and determine if we can test to see if two tags are perpendicular – and thus probably on the same cube.  2) a team member will also print out Home base so we can start including this in our physical tests.

I have started reading the paper assigned to us and which is linked on the Project page.  So far it seems easier to understand than some disserations I have read this semester!  I’ll post thoughts and notes applicable to our project when I am finished.

I will be giving Achilles to Catherine so she can start working with physical testing.   My desktop version of ROS has not been updated since December, so it is compatible with the version on the rover.  She will probably end up updating his ROS since the versions on our laptops are newer.  The lid supports have been broken numerous times since they are weak at the elbow.  The fact the rover and my cat had an encounter of the 83rd kind did not help.  It was brief, involved the cat leaping on the roof of the rover then running away, but the last glue attempt on the struts fell apart.  I am glad otherwise the rover remains functional.  I sent the stl file to Dr C and he said he would print some for us as replacements.


February 16, 2017

We discussed today the need to look at the code and see if we can modify the turning of the rover so that it is slower.  Right now it seems the rover turns quickly after calculating a new point to move to, and thus may miss targets because of camera blur.   I dug through the code and was looking at the random walk function and asked the others to look at it today. Speed – as a wheel setting for example – is not included in this function so perhaps we can incorporate some changes to manipulate it.

We also will be seeing if we can apply what we learn about agents and searching in our AI class to this project.  Some of the algorithms will not work in the real-world, but Dr Bhattacharya said he would also recommend a few papers that perhaps would.


February 10, 2017

There was a huge amount of information shared at this Conference.  However, it is clear the target audience for most presentations was expected to be composed of researchers, industry-members, and faculty in AI-related fields.   As undergrads in Computer Science, Catherine and I did not have the background to understand a great deal of the material presented.

However, we gained a greater appreciation of the wide variation of specializations within the Computer Science field, and within AI itself.  I admit I found I enjoyed the ‘softer‘ aspects the most, such as discussions of Ethics and Natural Language Processing  – soft being a term laughingly employed by one of the speakers, Dr Rosalind Pike.

We also met several researchers and faculty, contacts that we hope to cultivate.  One of the reasons we attended was to start networking with others in the field.

This slideshow requires JavaScript.

February 8, 2017

This morning we attended a presentation by Dmitri Dolgov titled Self-Driving Cars and the Future of Mobility.  Mr Dolgov was the technical lead on Google’s Self-Driving car project and is now head of the spinoff autonomous car division called Waymo.

He described in general terms the challenges they faced developing the self-driving car, such as ojbects that had to be detected and classified, and the learning involved as testing progressed.  An example was calculating the probabilities of path choices an object identified as an approaching cyclist might take and how the car would react in real time.  He provided a great many images showing object detection, annotation, and classification, but obviously no code was actually shared.  (That would have made our research project so much easier, though, yes?)

A video showing the first successful fully automated road trip in 2015 was shown, where a blind gentleman named Steve Mahan was picked up and delivered to his destination by a car that received zero input from the blind driver or observers and did not have a police escort while traveling on public roads.

He went to on describe future challenges to be faced before fully autonomous cars are introduced commercially.  Right now a self-driving car will accept – indeed, sometimes needs – intervention from the human driver to address situations programming has not mastered.  Current technology supplements the human driver but does not replace the human driver.   A recent crash involving Tesla was mentioned as a situation they hope will lead to improvements in programming, as well as addressing how cars might interpret input from outside entities such as a police officer directing traffic.

In the afternoon we attended several presentation sessions by researchers, including those focused on Multiagent Systems and Robotics.  The material was far above our level of comprehension but we still took notes and saved copies of the accompanying PDFs with the hope they will be useful in the future.

Since we were sponsored with the intent of attending the Diversity workshop, I made note of the observable demographic factors of those attending the conference. There is a pronounced imbalance. The majority of those attending were males of Asian descent, very few females were seen.  Hispanics, Native Americans, and African-Americans of either gender were rare indeed.  In the session below I counted 75 males, 8 women.




February 5, 2017

Workshop day.  We attended  Increasing Diversity in AI and enjoyed it immensely!   The purpose was to discuss the importance of using multiple viewpoints and ideas to solve AI problems and to explore challenges faced by members of under-represented groups in AI.

Speakers and topics were Ece Kamar, Microsoft Research, Directions in Hybrid Intelligence: Complementing AI Systems with Human Intelligence;  Luis Ortiz, Modern Applications of Game Theory in AI to Social and Economic Systems;  Judy Goldsmith, University of Kentucky, What’s Hot in AI Ethics;  Sheila McIlraith, University of Toronto, AI and the Future of Automated Programming;  Jennifer Neville, Learning in networks: How to exploit relationships to improve predictions;  and Marie desJardins, Finding Balance and Joy in an Academic/Research Career in AI.


I think I enjoyed the presentations by Dr Goldsmith the most, since I am intrigued by the ethical questions that arise when considering human-AI interaction.  I also enjoyed Dr desJardins presentation since I can identify with the challenges of balancing the needs of others with what I would like to pursue, and then discovering later what I wanted was really something else entirely..

We then were able to pose questions to a Career Panel featuring Puja Das (Apple), Ece Kamar, Judy Goldsmith, Jennifer Neville, and Sheila McIlraith.  My question was, given that the panel members were involved in hiring or PhD student selection, what role ageism played in their decision making, or what role it played in their workplace in general.   Most of the panel members seemed to think it really mattered more what a person was bringing to the table and how well they would fit with the team.  One mentioned that being older and not having school-age children that might pull them away from work could even be viewed as a plus!

Dr Ortiz ended by saying answering this question might end up being the topic of another seminar entirely.


February 4, 2017


We are clearly in the right place!   Our hotel is right across the street from the Hilton and we’ll be able to quickly reach our selected sessions.  The organizers have also employed an app called Guide to aid attendees, allowing them to search through available programs, review PDFs, and create a personalized schedule with reminders.

Today was Tutorial day – in the morning session we attended Learn to Write a Scientific Paper of the Future: Reproducible Research, Open Science, and Digital Scholarship as presented by Yolanda Gil,
Daniel Garijo, and Gail Peretsman-Clement.  We were somewhat disconcerted to discover it was not actually about how to write a scientific paper, but instead how to manage the digital version of a paper you have already written.  We learned about Data repositories, Digital Object Identifiers, how to cite data posted by others, and how to secure a license to protect our own metadata.

We then attended AI planning for Robotics, given by Michael Cashmore and Daniele Magazzeni.  This is where my concern that most of the conference was aimed at more experienced computer scientiests and not undergraduate students was confirmed.  Still, we listened and tried to follow given our level of understanding.

*Figure showing types of Licensure is from Creative Commons


February 3, 2017


Travel day.  3 hour delay in Minneapolis due to bad weather conditions in San Francisco.   Is California allowed to have bad weather??    This is the view from the plane as we approached the Minneapolis airport.  I tried to tell Catherine that the white stuff on the ground was imported beach sand.


February 2, 2017

At this meeting we met the final new members that have joined as part of our AI course.  To get everyone started with an equal (somewhat) foundation, we were asked to go examine the Swarmathon base code again and review all the functions so we know their purpose and how they interact with one another.

We identified some additional goals: short-term (or medium, actually) is to examine the current random walk search and consider other algorithms that might prove more effective.  I’ve been looking at the search source code and comparing it with what we have been covering in class recently – which is uninformed and informed searching – and making notes on what might be applicable in our project and what clearly will not work.

As stated in our Project blog, our long term goal remains to produce code that will effectively search for, detect, identify, collect, and return target objects to a  home base, aided by April Tags on the objects and at home base.  Ideally, the code will be good enough to win the Swarmathon competition in April.  However, this is a competition so we cannot post specific information in our blogs.

This meeting was short because we were encouraged to attend a seminar on Behavioral Cybersecurity offered by Dr Wayne Patterson of Howard University.  Howard is now offering a PhD program in Behavioral Cybersecurity, which sounds like an interesting variation of the purely technical field I am accustomed to reading about.

In the seminar, Dr Patterson explained the standard Cybersecurity curriculum has been expanded by including some Psychology courses.  The intent is not only to understand how someone might attempt to circumvent security applications, but also why.

Catherine and I leave tomorrow for the AAAI-17 Conference in San Francisco.  We hope to bring back materials that will assist us with our Swarmathon and Computer Vision projects, as well as network with faculty and researchers in the field.


January 26, 2017

Today we covered several matters and discussed blog guidelines as described to us in a mailing from CREU.

We have indeed received some new members from the Artificial Intelligence class so a short-moderate term goal is to again review the Swarmathon base code and gain an understanding of what the functions do, how they share data, what triggers implementation.   We are also encouraged to, as we learn about various searching algorithms and concepts in class, consider how those concepts might be applied to the Swarmathon code – another goal of moderate duration.   So far we have primarily learned about agent types, agent program types, and task environments.

Catherine is attending the Pipeline Conference today and we look forward to hearing about her experiences there.


January 19, 2017

The research group resumed weekly meetings today.  Dr Bhattacharya has announced to his AI class the option of joining the Swarmathon team in lieu of completing a number of required class projects, so I am hoping we will see some new faces in the coming days.

We updated everyone on conferences we would be attending and summer internships we have applied for, as well as what we did over the break regarding the rovers.   I’ll be attending the AAAI-17 but will probably have to decline attending the SIAM CSE17 in March because I would be away for a week and I will already miss a week of classes due to the AAAI conference.  In addition I would need to arrange again for a caregiver and I cannot continue to impose upon family members.  I had to decline the possibility of a Livermore summer internship for this same reason.  I am looking at local summer internships, however.

Josh C was also not successful in finding the keyboard control code.

There is a Swarmathon webinar at 1500 that we need to watch.  Hope we find out more about what we can expect in April!

January 10, 2017

The break went by too quickly, my classes start on the 17th.  Catherine and I have been working on arranging matters so we can attend the Association for the Advancement of Artificial Intelligence Conference on Artificial Intelligence in San Francisco from Feb 4 to Feb 9.    CRA approved some initial funds to attend the Conference and we have applied for additional funding through CRA/CREU to cover the remainder of our costs for travel, hotel, and meals.

December 20, 2016

Achilles said no to vacuuming.  I have not been able to find the keyboard control code in the Swarmathon code.  I need to check the Gazebo materials as well.  At this time, a user can control the movement of a swarmie from a computer using the keyboard.   To control the gripper, we have to use an Xbox controller.  We only have one of those.  This is not a huge priority because generally we are only manually controlling one rover at time, but should we ever wish to have multiple rovers controlled by team members simultaneously, updating the code to allow keyboard manipulation of the gripper would be nice.

I managed to squeeze a 4.0 out of this semester.  Hope I can continue that til graduation.

December 11, 2016

Exams are over and everyone is waiting on final grades.  I brought Achilles home with me.  I hope to become more familiar with the code on break, and maybe I’ll even be able to program him to vacuum.

December 1, 2016

We had a meeting today that documents most of what I did today.  I’m putting a micro-tool kit on my Christmas List because the one Josh C owns has certainly come in handy.

I plan to spend most of my break catching up on code changes and getting a deeper understanding of the entire Swarmathon project.  Ryan has been our go-to guy but we have to learn to stand on our own and move forward and do something meaningful with this opportunity.  He’s mentioned before that when he started, what he thinks was most valuable was just digging into the code, immersing himself in what was written, figuring out what it did, how it did it, and why.  Until we firmly grasp this, most of the code we write may well be shots in the dark and not perform as we hope because we lacked some fundamental understanding of what was going on under the hood.



November 17, 2016

Today’s meeting made me realize how much I still need to learn about the nitty-gritty of rover function and Swarmathon code.  Thanks to Ryan’s presentation I have a bit of a better understanding of how different parts of the code receive, share, and act upon data.  Before it was “oh, well, the ultrasound detectors ..  err, ahh, detect something disrupting their signal and magic happens and the rover changes directions away from the disruption source.”  We’ve been focused on writing code and reviewing code but did not really look at the abridge folder and function that much.

I have written a couple of flow charts — doodles, really – about things I’d like to see us add to the code, or see if it is possible to incorporate into the code and have it prove useful.  I have a MIPS assignment due that is taking most of my attention right now.    I think everyone is feeling pulled in too many directions right now as the end of the semester draws near.


November 14, 2016


I’m reading a few pdfs and articles about pose and calculating distance from camera to object.  I need to conquer my fear of math!  This is why I sought a minor in Mathematics, to prove to myself I could handle such concepts and calculations, not to mention that it aids in programming immensely.  I need to go back over the April Tag Library again to improve my understanding of what the code can do, and where we need to make changes and additions.

However, Casey, Josh C, Brittany, Donny, and myself met this morning and the grippers on Aeneas were not operational.  Previously Casey was able to use an Xbox controller to control the grippers’ position and angle on Ajax and Achilles.    Josh went to work with his tool kit so we’ll see what happens there.  It may be a wiring issue.  There has also been a ros update so we need to be sure all our devices are current.



Meanwhile, Brittany and Donny were able to see how to interface with the rovers – connecting to the network, adding IPs to /etc/hosts on each device, communicating with the rover in the terminal, etc.   So much of our work so far has been code or theory related, or focused on image processing.  Now we’ll be using the rovers more as we try to optimize our target collection.







November 6, 2016

Casey, Josh C, and myself from the Robotics group, as well as Catherine Spooner and Sam Cooper from the Human Tracking and Surveillance group attended the SNCURCS conference yesterday in Durham, NC with Ryan Fisher, Student Lead, and Professor Bogdan Czjedo.

Here Casey is answering a question about image annotation using OpenCV.caseyjoshcposter

We had a chance to listen to some interesting presentations by fellow undergrads.   One we attended had used object detection to identify a bee and then track how much entry/exit activity was taking place at a hive.  Sadly, the object detection portion had been developed by another researcher in an earlier portion of the project and detection was based on the general dimensions of the space a bee might occupy rather than any features of the bee itself.  Even though the focus for our detector has changed, I had hoped to discover potential methods for improvement beyond increasing the annotation pool.

This provided great experience for us.  I think we were all able to identify ways we could improve our presentations, explanations, and documentation for future events.



October 31, 2016

Worked on our conference poster for the last couple of days, submitted it this evening via email to Dr Bhattacharya for his recommendations for revisions.   Have been reviewing the NASA Swarmathon webinar recordings and going through the April Tag code.

October 24, 2016

Went to the MD today because I seemed to have gotten sicker since Thursday (AC in ScienceTech building went awry) and I do have pneumonia.  MD said I need to stay home for a week, evidently because this is a bit of a relapse and I did not seek treatment for the earlier illness.   I’ll ask Brittany to update me on what happens at the next meeting and meanwhile I’ll catch up on reading and looking at this new April Tag Library.  I updated Ajax on Friday with the updated ROS and tested it via simulation.

October 19, 2016


We have a Swarmathon webinar on Friday, but it is at our Cpp class time; perhaps we can arrange with the instructor to be allowed out, but it will be recorded if we cannot.  I registered, just in case.  I’ve also requested travel funding to the AAAI 2017 Diversity Workshop in San Francisco scheduled for February 4-5, 2017.   I’m required to submit an interest statement in order to qualify for funding to attend The Pipeline Workshop on High Performance Computing taking place at the University of New Mexico from January 26-27, 2017, so I’m working on that as well.


October 12, 2016

The area is recovering from Hurricane Matthew so our team has not met this week.  I’ve been without power since Saturday the 8th but it came on this morning a little past midnight so I got online asap and submitted the abstract for SNCURCS.  Power has been intermittent as power crews move through the neighborhood.  Hopefully I’ll be able to update Ajax next week and the team can start designing our presentation poster and decide what information to include.

Glad this tree was not taller!  Small branches luckily did not damage my roof.







October 6, 2016

Submitted a report to Dr Bhattacharya as part of our CREU progress tracking, can see Project blog for more information on that.    Spoke with Dr Bhattacharya earlier today about a potential addition to our team, a gifted classmate who has expressed an interest in the Robotics internship and who served with us on it during the summer.  He’s been encouraged to start attending our meetings and I think he’d be tremendous asset.

I returned Ajax to Room 213 this morning and I’ll be updating his software to the newest ROS version on Friday if I have time between meetings and class, otherwise on Monday.  Meanwhile I’m reading the early release of the newest OpenCV 3 document.   We’ll have to determine if any of the OpenCV 3 coding will even be compatible with the Swarmathon code, which was based on OpenCV 2.4.

The NASA competition is in April and I’m hoping our efforts allow us to secure first place and continue the winning legacy of the previous Cooperative Robotics team at FSU.  With improvements to localization and our work on this object detector, I think we’re off to a good start.


October 5, 2016

Donny came by the office today so we could install Ubuntu on his new Lenovo Yoga 900S.  Casey and Joshua had tried to help him install Ubuntu on the Aspire earlier this week to no avail so he returned it.  However, during installation when we came to the point where normally one is prompted to indicate if they wanted to dual boot, wipe the current OS, etc etc, it instead skipped to a partitioning menu, but no available partition or drive was found.  Unexpected.  Tried again. Same issue.  Normally Ubuntu will generate its own partitions, but we decided we’d go ahead and create a partition for Ubuntu to install to and hopefully the new partition at least would be detected.  So we went into Windows, generated a 50GB partition we thought would be more than sufficient for Ubuntu and a swap, then booted with the install usb.  Again, no available drive found.

So Dr Google was consulted and on the Lenovo forums it was discussed that  one cannot install Linux on this type of laptop because the “SSD is locked in a proprietary RAID mode that Linux doesn’t understand” and that the devices are “locked as per our agreement with Microsoft.”  Donny was frustrated by this, since this is the third device he’s tried to install ubuntu on. We then tried to install to an external but that did not work either.  He’ll call Lenovo Support and confirm what was stated on the forums and either get a walk-through on the installation, or he’ll return the laptop for a different one without this restriction on it.


October 2, 2016

When submitting the abstract for the conference it requests a length not to exceed 2o0 words!  The published guidelines had stated up to 300.  Argh. So. Had to cut 97 words.  I can appreciate being concise, but this was almost painful.  Also the submission deadline is now listed as the 12th.

Downloaded the new ROS/Swarmathon code from github.  Fun to see the grabbers on the simulated rovers, field of view is certainly at a much lower angle as a result, but in the sim, at least, it appears you can see the grabbers interact with an object.  Guess we’ll see soon if in real world testing the view is the same or if we’ll be picking up the item blindly and only be able to ‘see’ it once it’s lifted back into the camera’s view.

Donny purchased an Acer Aspire R15 so hopefully we can get his software installed.

On a personal note, Mom is being treated for pneumonia and I suspect I have the same thing.


September 28, 2016

Donny came by the Student Center where I was doing office hours for my commuter organization.  We worked more on his notebook to see if we could research the error messages we were getting.  I think we are just asking the device to do more than it was created to do.  He’s exploring some possible laptop purchases.  He has good desktops at home, so a basic laptop should be sufficient for internship/school purposes.  No big GPU, etc, needed, since we have other laptops with dedicated GPUs that can run our programs in the field.


September 26, 2016

Ubuntu is working on the external drive now and thus far no errors have been encountered.  OpenCV 3.1 was reinstalled, annotations recreated without issue.  I had saved the old files, but started testing the software and just ended up doing them all.  Now need to generate more test images so we can get a couple thousand at least.



September 25, 2016

Tried using Ryan’s rescue disk on my Ubuntu but it ended up just giving me a text-based interface and would not recognize the mouse or touchpad input.  Backed up data, cleaned the second hard drive, reformatted.  Tried a fresh installation of Ubuntu and I get a series of error messages.  I can still use my desktop to process annotations, but that’s not going to be of any use when I need to go outside with the rovers and take more images to add to our training bank or to our validation bank — which cannot contain any of the images used in training.  Will try external drive next.

September 18, 2016

I freely admit little progress this week due to time constraints.  Mom is out of the hospital and in a rehab facility.  I end up spending most of my non-class time there.  She’s still spiking a temp and this long post-op she really should not have a fever that high just as an inflammatory response.  Lots of edema, no purulent drainage.  As a bonus, the AC in the house died and the car has a coolant leak.  Not really internship related, but certainly has impacted my participation recently and I have not resolved all of the Ubuntu errors, even with a fresh installation.

September 16, 2016

Had scheduled to help Donny with his notebook but he ran into some issues and we’ll do it at a later time.

September 12, 2016

Spent the day at the hospital, mother received a knee replacement, so focus has been divided.   During surgery my Ubuntu decided it would be problematic again.  I’d switched to Windows to handle a project for another class, then returned to Ubuntu to work on additional annotations and it would not implement any annotation commands even though directories and files seemed intact.  A reboot resulted in the return of an old error – black screen with drum-roll to indicate the log in screen had been loaded.  There have been some recorded issues with my GPU and Ubuntu and I had been using nouveau vs proprietary drivers.  Since we will not be using ROS/Swarmathon code for a bit, I may just reinstall Ubuntu tonight and only load the OpenCV that I’ll be using to annotate.  I did get logged in and load the desktop on an old kernel and retrieved some pertinent folders, but this is a recurrent problem and a fresh install will remove whatever ill-advised packages and software I may have downloaded and is now impacting necessary applications.


September 8, 2016

I’ve been continuing to read the Blueprint e-book.  There have been instances where it generates a ‘green’ saved box around something I never actually drew a box around.  I’ll have to be careful not to click in the image until I am ready to draw.  I evidently forgot that we had pondered updating the annotation code to allow ‘do overs’ when the box we draw is not quite what we intended.  The default code allows a keypress of C to save the image with drawn boxes, and N to save and then load the next image.  As Ryan mentioned in the meeting today, if you are on image 99 and mess up, there is no way to correct it, you just have to start over.  He recommended for now just separating the images into folders of ten or so to lessen the impact of a foul-up in image marking.

Adding a loop to allow an image to be reloaded if an error is made would be a good idea.  I’ve added it to my list of things to do, which is getting a tad long.



September 7, 2016

I posted a StrawPoll to get feedback from the entire group on possible meeting dates outside the formal meeting with Drs Bhattacharya and Czejdo.  Early results look like MWF late morning/early afternoon.


September 3, 2016

Worked with the annotation software to filter through some of the images obtained in our previous internship.  These images are of Ryan manipulating the cuboids with april tags adhered to them, placing them in various configurations.  I still need to ask him to send the images we took in the parking lot at school since we included frames we intended to be negatives as well.  I’m trying to get a good ‘box’ around the cube while limiting how much background is included.  

My laptop is still experiencing some difficulty due to running OpenCV 2.4 and OpenCV 3.1.  The latest kernel update also seemed to generate some problems with my GPU, so I had to revert to using nouveau.  I’ll research this a bit and see what I can do.  I’m still very much a novice when it comes to Linux/Ubuntu.


September 1, 2016

Had meeting today, can see Project blog for that information.   Cooper, Spooner, and myself remained behind to try to install Ubuntu on Cooper’s laptop using a DVD.  While checking drive space in Windows seemed to indicate he had sufficient space, when trying to install Ubuntu it would not permit partition creation.  We settled on two ideas:  he could make an external recovery usb and use his laptop’s recovery partition for his Ubuntu installation, or he could purchase a dedicated external hard drive and install Ubuntu on that.    I’m thinking option 2 might be best so we don’t create more problems than we are capable of handling at this point!

My annotation software was installed during the summer so I’ll start working on the trainingImages folder Ryan previously mailed to participants.  I’ll need to obtain the images we took in the parking lot at the end of summer.


August 25, 2016

Still reading posted materials.  Did read the ROS material posted for our group.  Have confirmed that ROS and Gazebo are running fine on my laptop and desktop.  I have OpenCV 2.4 and 3 installed right now but have not worked with it since the summer.



August 18, 2016

Can see Project blog for overview.  I’ve already created a Piazza account and a blog, so I’ll be reading the materials Dr Bhattacharya has posted on Canvas, the Learning software Fayetteville State employs.  Some of our group do not have personal laptops, so I’ll be dusting off my old Asus and giving it to Brittany.