CS-7646 | Machine Learning for Trading (ML4T) | Summer 2022

August marks the end of my second semester at Georgia Tech. I decided to take CS-7646 Machine Learning for Trading (ML4T) for my Summer 2022 course due to it’s positive reviews and to get experience with Machine Learning. The course was created by Professor Tucker Balch, and instructed by Professor David Joyner.

CS-7646 is a mix of lessons related to investment trading topics, as well as machine learning algorithms. The course is project heavy, and includes a multiple choice midterm and final. Since summer is a condensed semester, a project was due every week, which put constant strain on completing the assignments on time. Overall, the topics of finance and how we could use machine learning concepts to determine investment strategies were interesting and were good primers to introductory topics in both fields.

Finance topics include stock market analysis (technical analysis vs fundamental analysis), market indicator evaluations(bollinger bands, simple moving average, rate of change, etc), market theory and concepts (efficient market hypothesis), and more. Machine Learning topics included linear regression and categorical learners, optimizing, implementing, and evaluation of learners such as decision trees, random forests, and bagging.

The course projects were written in Python, with heavy use of NumPy and Pandas libraries to deal with large datasets of financial stock data.

Below are brief summaries of the 8 projects we completed in this course.

Projects

Project 1: Martingale

Conduct probabilistic experiments and write code to analyze the Martingale betting strategy involving an American Roulette Wheel.

Project 2: Optimize Something

Use python to optimize a theoretical financial investment portfolio. Find the most optimal allocation of investments to maximize gains, and evaluate portfolio metrics including mean, standard deviation, cumulative returns, sharpe ratio, and more.

Project 3: Assess Learners

Implement and evaluate a decision tree learner, random tree learner, and ensemble learner using bagging.

Project 4: Defeat Learners

Create a dataset that is optimized for linear regression learner, and a decision tree learner.

Project 5: Marketsim

Simulate market trades and return portfolio values and metrics over time.

Project 6: Manual Strategy

Create a strategy manually without learners that would give an optimally max portfolio returns. Develop and research technical indicators that can be used along with a learner to make buying decisions.

Project 7: Q Learning Robot

Implement a model free reinforcement learning algorithm called Q-Learning. Implement a Q-Learner with model based Dyna Q to solve a robot navigation problem.

Project 8: Strategy Learner

Use indicators along with learner of choice to create a trading strategy, and evaluate portfolio performance against a manually traded strategy.

Summary

In summary, ML4T at times felt like a grind, with a project or test due every week. Some projects had a Report component which added additional time outside of coding. In terms of learning opportunities, I came out of the class with financial and machine learning literacy I did not have going into the class. I have a stronger understanding of common market terms, and an introductory understanding of regression and categorical learners, how to implement them, and how to evaluate their performance. I do not feel as if I can implement my current knowledge of the stock market into a productive strategy that would maximize gains, as learning these new topics exposed the nuances necessary to understand the market and the multitude of indicators investors track in order to make business decisions.

I am happy to have had more opportunities to practice coding in Python, and using NumPy and Pandas.

CS-7638 | Robotics: AI Techniques (RAIT) | Spring 2022

Choosing my first graduate school course at Georgia Tech required time and considerable research. With so many subject matters and difficulty levels to choose from, I had to conduct extensive research to pick the right one, as this first impression would set the tone for the rest of my graduate education. Another factor that I had to take into account was Georgia Tech’s policy that requires two B’s or higher in two courses the first year to maintain enrollment. After many hours cycling through reddit, google, and OMSCentral (a student run course ranking board website), I decided on taking CS-7638 - Robotics: AI Techniques (RAIT), an average difficulty course with great reviews.

Syllabus

The course teaches different artificial intelligence algorithms that are commonly used in robotic systems such as self driving cars, or robotic vacuums. The lessons are structured to teach us how to program all the major systems of a robotic car based on lectures from the former leader of Google and Stanford’s autonomous driving team, Dr. Sebastian Thrun. The course itself is instructed by Georgia Tech’s Dr. Jay Summet and various TA’s.

Learning Objectives

The objectives for this course were:

• Implement filters (Kalman and Particle filters) in order to localize moving objects whose locations are subject to noise.

• Implement search algorithms (including A*, Dynamic programming) to plan the shortest path from one point to another subject to costs on different types of movement.

• Implement PID controls to smoothly correct an autonomous robot’s course.

• Implement a SLAM algorithm for a robot moving in at least two dimensions.

These objectives were met through Problem Sets, Projects, and Exams. Programming in this course was done in Python 3. Math used in this course was primarily probability and linear algebra.

Projects

The largest bulk of information learned and what accounted for 60% of our course grade were projects. Projects contained some starter code and a doc sheet with requirements that our program had to follow. The goal for each project was to implement filters and algorithms we had learned from lectures using Python. Although time consuming, completing these projects solidified robotics AI techniques and was thoroughly fulfilling.

Project 1: Kalman Filter

In the first project, we were tasked to implement a Kalman Filter to estimate locations of incoming meteorites, and to navigate a turret towards the moving meteorites to destroy them. The course provided the working environment, with test cases that included a random set of incoming meteorites.

Kalman Filter is an algorithm used to estimate system parameters, even ones that are not observed. In our project, we used Kalman Filter to take in a noisy measurement input to estimate the future state and velocity of a meteorite.

Image from: https://thecustomizewindows.com/2019/03/kalman-filter-to-stabilize-sensor-readings/

Above are the general steps used in a Kalman Filter. In summary, the Kalman Filter works in two phases, the Prediction Phase and the Measurement Phase. In the prediction phase, the Kalman Filter produces estimates of the current state variable using the initial or previous state. In the Measurement, or update phase, new sensor measurement inputs are used along with our new predicted state in a calculation to predict the updated state. The average is weighted using a calculated Kalman Gain to determine which value (measurement, or predicted) we trust in more. The new updated predicted state is then fed back into the filter as the previous state until we get closer and closer to the actual state.

Having estimated the meteorite’s state of location and velocity using a Kalman filter, I was then able to use knowledge of our grid world, the predicted state, and trigonometry to guide our turret towards the meteorite to destroy it.

Project 2: Particle Filters

In this project we implementing a particle filter to estimate locations of planets in a solar system. We populated the grid world with N number of particles, assigned each a weight based on the particle’s gravitational reading from our sensor compared with the planet’s gravitational reading, and resampled our population with particles of higher weight being chosen more frequently. Particles with similar sensor readings to our planets are assigned a higher weight. This algorithm is survival of the fittest, where measurements that are far from actual are more likely to be removed from the population through resampling.

Project 3: PID Control

For this project, we implemented a PID controller to control a flying drone’s hovering altitude and location. Given a goal altitude and location, we needed to control the drone’s RPM to hover at the target location and altitude for a specific time limit. We implemented P, proportional control to reduce our error (in this case difference in position from goal) by controlling the drone’s propeller RPM in proportion to distance away from goal. This parameter needed to be tuned until we were getting our drone close to our goal.

This resulted in our drone oscillating around the target altitude, where the drone would approach the target altitude, but overshoot it, then correct itself. To account for this, we needed to incorporate the derivative of the error in our calculation. The derivative produces an output based on the rate of change of the error. This means as the drone approaches the goal altitude, the value of the derivative approaches 0, smoothing down the approach towards goal, which prevents oscillation and overshooting. To account for system errors, an integral of the error is added to the calculation. The integral control will continuously increment or decrement the controller’s output to reduce the error to 0.

The PID controller was simple to implement. The greatest challenge was fine tuning each parameter to minimize error. This could be done manually, or using a Twiddle Algorithm.

Project 4: Warehouse Search

The fourth project involved controlling a robot around a warehouse to retrieve and drop off packages. We were to implement search algorithms, mainly A* Star and Dynamic Programming for stochastic cases where the starting position is unknown. The warehouse world is a discrete grid, with each position having varying costs. We had to find the most efficient path to the package, and to the drop zone.

A* search algorithms uses a known starting position and expands outwards and checks whether neighboring spaces are the goal, all while keeping track of accumulating costs. A* Search algorithms also use a heuristic function as a map to guide its decision.

Dynamic programing accounts for the stochastic nature of the world by calculating optimum paths to the goal from any starting position. Implementation of Dynamic programming to find the optimum path from any position to a goal has the same structure as A*star, with some extra accounting steps for a value grid that is used to count backwards from the goal to calculate the cost of reaching the goal from any position. These costs are then used to assign the action that the robot needs to take.

Project 5: SLAM (Synchronous Localization and Mapping)

In project 5, we had to localize our drone while navigating the drone to a known treasure position. With an unknown map, we could not localize using traditional filters or methods. We implemented an online SLAM (synchronous localization and mapping) algorithm to localize our robot while mapping out positions of landmarks in our map. We use sensors of landmark positions and movements to update our state matrix with the location of our drone and of the landmarks. We do this by considering the measurement and movement constraints and doing simple matrix multiplication.

Conclusion

CS-7638, Robotics: AI Techniques (RAIT) was a great first impression into the program. I highly enjoyed the course topics, the way it was presented, and the support received from my colleagues and TA’s. At times, the projects seemed impossibly challenging, but the process of overcoming unknowns through days of research and debugging reinforced my learning of the implementation of the algorithms and was fulfilling. Aside from learning algorithms and python, It was a good learning experience for managing and allocating my time between grad school, a full-time job, and family and friends.

Georgia Institute of Technology

Why grad school?

Navigating my early career after undergrad has been a relentless and confusing journey. The beginning of my professional career was met with uncertainty and anxious thoughts of whether my degree in Bioengineering was the right choice. I spent the time exploring various professions that I thought my degree prepared me for, only to realize I didn’t enjoy working in research (molecular biology) laboratories very much. I had to reevaluate my career trajectory and examine why it wasn’t aligned with the opportunities I had hoped my Bioengineering degree would provide. After learning and gaining working experience in web development, I have come to the conclusion that I enjoy using my creativity to program solutions with code. Though I took courses in computer science as part of my undergraduate program, I still have a lot to learn. I decided to pursue a master’s degree in computer science to satisfy my curiosity to learn more about this field and to pick up useful tools along the way. I matriculated at Georgia Tech in Spring 2022.

Why Georgia Tech?

Online Distributed Learning

Aside from being a renown university for Computer Science, I heavily considered Georgia Institute of Technology for being a leader in online distributed learning. One of the criteria I had when considering graduate schools was the possibility of online learning. I wanted an accredited program where I could pursue a degree without halting my industry growth. I also have bills to pay 🙃. In the past, online distributed learning has been met with jokes of being ‘less than’ when compared to on campus learning. This view is a misconception, but may have been propagated by the proliferation of for-profit, unaccredited institutions. With the pandemic and the widespread roll out of ‘Zoom’ school, I hope people could see the utility and legitimacy of online distributed learning. Since 2014, the online master's of computer science program at Georgia Tech has grown a positive reputation with excellent reviews. I appreciate that the online course selection is expansive, and is a direct equivalent of on-campus courses, taught by the same campus faculty.

Cost

Another big factor in my decision was cost. I had also considered Johns Hopkins University’s program, as well as University of Pennsylvania’s program, but in addition to poor reviews, the price for tuition was shocking. After being scarred from paying off my undergraduate tuition, I was in no hurry to put myself back in debt with student loans. Johns Hopkins and UPenn’s programs can run you $40,000 - $60,000, roughly $4,000 - $5,000 per course. This value is ridiculous compared to Georgia Tech’s tuition of $7,000 - $8,000 for the entire program!

Community

Finally, the last factor I heavily considered when choosing a graduate program was community and resources. Georgia Tech has a large and growing network of alumni who have completed the online master’s program, as well as an active reddit community and slack channel. Since this program is online, having a sense of community and camaraderie among my peers was important. After finishing my first semester, I am happy to have chosen Georgia Tech. The support from TA’s and my peers on Slack / reddit have been a large component of my success and motivation to continue.

Application Process

The application was a straight forward process. I gathered all required documents, letters, and statements to submit with my application by the deadline. No GRE was required. The decision was returned a few weeks after the deadline.

Requirements

• Undergraduate degree in computer science or related field

• Cumulative GPA of 3.0 or higher

• Transcripts

• Three letters of recommendation

• Statement of Purpose

• Supplemental Objectives Statement
  

Closing Thoughts

The program requires completion of 10 courses of your choosing with specializations offered for Computational Perception & Robotics, Computing Systems, Interactive Intelligence, and Machine Learning. I have completed my first semester taking CS 7638 - Robotics: AI Techniques and have had a wonderful time. I have thoroughly enjoyed my first semester and am excited to continue through the Computing Systems specialization. Individual course reviews and summary posts will come.

Go Yellow Jackets! 😁