Category Archives: Software

AI Nano degree update – Finished Term 1

With much relief, I am happy to say that Term 1 (out of 2) is now complete. Since my last update, I went through two more projects. How did it go?  Overall satisfied with how much I learned but the ending was rather boring and I couldn’t wait to complete the assignments and move on to the next term.

Project 3: Implement a planning search sounded very excited. Planning AIs is a big part of AI systems but the way the topic was presented in the class was just absolutely terrible. It was presented in a strict academic fashion and I walked away from the project feeling very dissatisfied. I spent most of my time not on learning and experimenting with the concepts but deciphering rather confusing code setup and data structures from AIMA code. The actual solution was a very straightforward copy/paste exercise that taught you close to nothing.

After thinking about this more, I don’t per se blame the materials or the Udacity. The issue here is that the subject is very broad that doing it as a 3-week exercise most likely will leave you dissatisfied no matter how you present it. To do a full project from the ground up, coming up with planning language, possible states, etc is an involved exercise. That’s where I think Udacity was better off leaving this as an optional part of the course and spent more time on search techniques or tackled more advanced adversarial search techniques like Monte Carlo Tree Search.

Project 4 was not that much different in how much it disappointed. The topic itself once more very sophisticated and exciting: HMMs. I am not sure how much I learned from the lectures and instead relied on other resources to start to understand how would one go about using HMMs. After doing the project I realized that the lectures were incredibly disconnected from the project. One could do the project without looking at the lectures at all. The videos for part 4 should just be scrapped altogether. They add no value.

 

I don’t want to end on a negative note since overall I enjoyed Term 1. The first two months were incredible, exciting and I learned a lot. The second project especially, search techniques for solving problems, was by far my favorite and one of the main reasons why I took the course.  The udacity’s system of slack rooms and office hours combined with the projects were really useful for learning and I am glad to be able to continue on.

For now, I am taking couple weeks off until Term 2 begins. Term 2 focus on more practical uses of AI and deep learning, culminating in students selecting a single specialization out of these three areas: voice user interfaces, natural language processing, and computer vision. I am leaning towards computer vision as I have always wanted to learn more how such systems operate. Although I do wonder if voice user interfaces would be more useful from a career perspective. Perhaps I will do image processing and then do voice user interfaces specialization as an add-on. Let’s hope Term 2 starts with a bang like Term 1 did and stays that way!

AI Nano degree update – Project Two

This is the second post in the Udacity’s AI Nano degree series. Taking the course is my attempt at learning and re-familiarizing with AI / CS concepts.

Project #2 tasked the students to implement an AI agent that is capable of playing a game of isolation. The only exception from the usual isolation game was that the moves had to follow a chess knight’s pattern. The course provided the board implementation and we had to write AI agent that is capable of searching for optimal moves through the board with the goal of defeating an opponent.

Sample isolation board. Visited squares are in the grey/black shade.

Initially, the game itself was not something to get too excited about. But when the implementation of algorithms started that’s when the fun picked up! Let’s start at the high-level definition of a problem. Given a board, it is not obvious what should the next move be in order to guarantee a win. One could run simulations and try to find it, but the search space can be too large even for the small boards (7×7) making it impossible to solve by a brute force alone. Instead, AI agent should focus on optimizing how it traverses the game tree by doing two things:

  • Come up with a way to evaluate the score for the board positions. A winning board has a score of infinity, a losing board has a score of negative infinity, anything in between should have a score that correctly determines how favorable the board is for winning or losing.
  • Iterate the possible solution tree in a way that is fast. You want to throw out the boards that are unfavorable to you, or select ones that were known to benefit you.

For the first problem, coming up with a way to value a board, means defining a heuristic function for the board position. The strength of the heuristic function is the difference maker in the AI agents. It requires a balance between being accurate but also fast to compute so that your algorithm can evaluate many boards during a single turn. If you make a heuristic function that is too complicated, then AI agent will be slow and with certain rules where time limits are enforced will lose.

For the second problem, iterating the possible solution tree, there are approaches such as minimax, alpha-beta pruning to optimize your move selection. Mix in iterative deepening search and you got an effective agent.

The heuristic functions I tried out where giving a score to a players position that indicated how many open moves the player had vs the opponent (the more moves over the opponent you have the better) COMBINED with how close the player was to the open positions. The idea here is not to get trapped. Another variation of the above I tried was staying as far away from the walls as possible. That one turned out to be a good heuristic, but not as effective as staying close to the open fields. And lastly I tried one combination where I combine staying away from the walls with staying close to the open fields and the results were still less than just staying close to the open fields.

All in all, it was a successful implementation that beat the baseline score set out by the project’s creators. Now I am trying to decide what to do with the agent and see what can be added to it so that it can participate in the competition against the agents of other students.

Some observations from going through the exercise:

  • Visualization is the king. Visualizing the board positions and move calculations really helped me discover bugs in my implementation. I should just always start with the visualization when working on the problems and go from there.
  • Iterative deepening was somewhat unintuitive at start. It is amazing how much it helps to find the solution faster without going too deep into only certain parts of the tree.
  • Alpha-beta can be a bit confusing at the start and you definitely need many manual/on-paper implementations to see why it is effective.

Some things that I did not implement as part of this exercise that still need to explore:

  • quiescent search. This one is a mystery and something that I will bring up on the course forums. I have read book materials on it and other online resources but something tells me that until I will try to implement it, it will not fully make sense.
  • Monte Carlo tree search roll outs. I really want to implement this one and see how it would make the AI agent better. Seems like it is a big part of any game playing AIs and making them effective.

This was fun. On to the advanced search concepts and pacman lab!

AI Nano degree update: Project One

First project required to write an AI agent capable of solving a Sudoku puzzle. The key goals of the exercise: familiarize students with concepts of constraint propagation and search for solving problems.

I would summarize constraint propagation as follows. In abstract, when you have a function that needs to pick a solution given multiple of choices, it can narrow down the answers by coming up with strategies that eliminate the subset of given solutions until a single solution remains or the number of potential solutions is smaller than the input solution space.

In Sudoku project, the three strategies for eliminating the solutions employed were as follows:

  • straight up elimination – find boxes that are solved, and then remove that box’s value from its peers (rows, cols, squares)

  • only-choice – given a box with multiple possible digits, for each of those digits, see if it is not present in other unit’s of that box thus making that number the only viable choice for the box

  • Naked twins – sometimes you have units that have two boxes with the same possible solutions, so that means those two squares will have either one of those digits. Thus those digits can be eliminated from other units of the box. There are variations of this called naked triplets, and so on. Twins seemed to be the most effective.

Now you can imagine that one can run through the solution elimination sequence in a loop, with each pass applying all the elimination techniques. You stop looping if none of the techniques are reducing the solution space further. You are stalled.

What now? Well, brute force search. Pick un unsolved box, and iterate through its solutions, in each pass applying the constraint elimination sequence until you either solve the problem or you stall. If you stall – the initially picked solution is not a good choice, move on to the next until solution is found.

The project was a very fun exercise. I never was much into Sudoku before so at the very least it gave me an excuse to try the puzzles out. It quickly became a fun exercise of finding patterns. And once you mix in writing Python code to solve the problem automatically, it was nothing but a delight.

Here is a screenshot of AI agent in action:

AI agent in action solving a sudoku puzzle
AI agent in action solving a Sudoku puzzle

To generalize the idea: when you have a function which has a set of possible solutions to choose from – go ahead and think through how you could constrain possible solutions to reduce the search space. Then brute force search through each until the answer is discovered.

Besides a great warm up into search, the first project also gave me a great intro into Anaconda, “Leading Open Data Science Platform Powered by Python.” Think of it as a Python environment that is loaded with data science libraries and tools. If that is not enough, it can “containerize” your Python environments that are entirely isolated and across machines/platforms. You can setup a Python 2 environment and Python 3 environments, load it on the same machine, and neither will impact each other. And again, not to mention that it comes pre-installed with a variety of data/ml related packages.

On to the second project, which goes much deeper into general AI ideas around search and advanced game-playing techniques. I am done with that project two and should have a write up for it shortly.

Machine Learning update – Feb 2017

In the last update on my machine learning journey, I had just finished the Udacity’s intro and started with the Coursera / Stanford Intro to Machine Learning. I am happy to say that this course is now complete as well!

It feels slightly surreal to reach this point. When I first setup my plan for ML, Coursera’s course was something I had marked as being challenging and a “maybe if time allowed.” The reviews and the feedback mentioned how great the course was, but also many people seemed to drop off at the neural network chapters. Essentially I had my doubts about being able to finish it on time while doing it part time. There is no time limit to the course, and you could transfer to the next cohort, but I wanted to make sure I did it in the same session I had started. Once you start delaying an online course, there is a chance you will delay it indefinitely.

In retrospect, the course was indeed challenging but not as bad as I expected it to be. The hardest part was to get comfortable with Octave environment and translating lecture notes and formulas into matrix equivalents. I am quite happy that I stuck to the end, and with 100% grade to boot.

If I had to compare the intro courses from Udacity and Coursera, I would still recommend Udacity to start and then use Coursera to augment and deepen the understanding of the basics. I had quite a few “aha!” moments when taking Coursera’s course, but Udacity makes ML more practical and attainable. I thought it demystified the Machine Learning field. After taking the course, you see the application opportunities and the landscape which you should further study. Perhaps best is to combine the two classes – they are drastically different – and learn and compare the concepts between the two.

What’s next? Feb 16th I am starting AI Engineer Nano Degree on Udacity. The same feeling again, a bit daunting and challenging. Hopefully, I will hang in there and power through it. I am sure to post the update as I go.

Before the course starts, I am taking a quick detour back to stats and statistical analysis, to make sure I grasp the basic concepts of analyzing data. Trying to go deeper into kernels and data sets on kaggle.com, familiarizing myself with pandas framework, etc. Basically having fun before AI degree ruins it all.

View story at Medium.com

A start of machine learning journey

Early in September, I started taking a course on learning. Essentially it is a course on how to be a better learner. Learning about learning might sound silly, but it was a great course with many great strategies to employ when trying to master new material or acquire new skills.

As part of the course, we had to pick a project that we will use to apply the techniques we were learning. The concept made a lot of sense. In my experience, the best way to learn a practical skill is to combine the theoretical knowledge with practical work, so the project seemed very appropriate.

My project was to take and finish a course on Machine Learning. I knew close to nothing in this area, and it is a field that is hot in software engineering. ML being new to me, it gave me a chance to program using techniques that are completely unknown and that makes things a bit frustrating but also a lot of fun.

The course I went with was Udacity’s Intro to Machine Learning course. So far the intuition to pick that course is proving to be correct. I finished it ahead of my planned scheduled. It took me just a tad over two months, while mostly studying on weekends and occasional early morning.

The most fun part was applying the skills learned from the course at my current job. We do some video processing tasks and such things have been notoriously tricky to estimate how long they will take to complete. With ML, and more specifically regression analysis, it was a breeze to build a model that gave excellent predictions on processing durations. Some of the predicted times were within seconds of the actual times, most within minutes, which was more than sufficient when you consider the processing could last anywhere from 20 to 40 minutes (with some outliers of course shorter or much longer).

My goal was to apply the techniques in some capacity by February 2017, and being able to do that so much earlier was a big boost and motivator to continue going strong. Actually one of the learning course main preaching points was to use what is being learned right away, even if you don’t feel like you know what you are doing. It just strengthens the knowledge and right away deepens your grasp of the concepts that you are learning.

I am highly recommending udacity’s course for the others that might like to start ML journey themselves. It is not very heavy on theory, although one should use the topics presented to dive into more theory online. The examples and mini projects they present are really great, interesting and informative. If you know a bit of python you are pretty much ready to go. Knowing some of the advanced math conceps helps to understand the course better, but it is not necessary in order to use the techniques.

What’s next? I am happy to share that I got accepted to AI nano degree. It does not start until February 2017, so in the meantime, I am taking another ML intro course, this time Stanford’s Machine Learning course which I debated to take before picking Udacity’s option. Stanford course is great, but presents much more theory and is a bit more “drier”, more pedantic. I am on a week 3 now of the course, going ahead as far as I can before Neural Network weeks. That area might prove to be very complicated so it will be good to have as much cushion as possible for quizzes and learning.

 

That’s it for now. I can’t wait to see where this ML journey will bring me. Hopefully, I will continue to deepen and strengthen my practical skills and start applying it in everyday life with regularity.

Couple minutes with Google App Engine

I am an avid reader and on average can go through 2-3 books per month. I created a simple site (source) which runs on Google’s App Engine to help keep track of the books being read. Long story short, one of its features stopped working recently and I kept on delaying investigating the cause until tonight.

What broke was a page which showed the latest books a user marks to be read, or starts / finishes reading. When a user selects a book to read, in addition of all the logic that runs for such request, a job is queued to add that book to the “reading” list. I knew right away that job handlers are running into some problems. I figured that it had been a long time since I checked the Google App Engine, that bug was a good excuse to check things out and see where GAE is these days.

I am so glad that I did. Debugging the issue was a breeze because of the logs that are available in the app engine’s administrator console. They really do a good job of making the information available to the developers. As I said, I suspected that job handlers are having issues and logs confirmed just that:

book_toobig

Too large? How can it be too large? I only store the latest 20 books for each “to read”, “reading”, and “finished” lists! Well, after looking at the code I realized that I had a bug there and no truncation was occurring once more than 20 books had been marked. So eventually that list got too big to be saved as a regular data store item.

Once the bug was fixed, the tasks continued to fail to run but this time with a different error:

book_str_error

Messages like these are extremely vague, but luckily the “str” gave me a clue that the issue might be an empty string I was returning in the task handler code. I implemented the queue functionality when it was still in the early days of GAE labs. Certain functions and behaviors have changed since and removing a return part solved the remaining issue. And voila, the summary page is back online:

booklist

It was great seeing the page and the variety of subjects that I get to read about. Some comedy, history, philosophy, self-development, and software – quite a nice mix!

I love evenings like these. A long neglected bug is fixed, couple things learned (e.g. I had no idea that GAE updated their docs with Jinja2 instead of Django, so off I went reading up about Jinja2), and a piece of nice functionality is back and available.