ML Launchpad Final Writeup

Grady Forsyth, Jack McKelligon, Kyle Gragnola

INFO 4700: Senior Capstone

Introduction

In this project, we aimed to bridge the gap between complex machine learning (ML)

concepts and learners new to the field by developing a simple facial recognition algorithm demo.

With a focus on making the code accessible, we recognized the growing integration of machine

learning technology in daily life, especially among young adults. Our initiative involved

constructing a ML algorithm, designing a user-friendly web interface, and testing the product to

ensure its educational effectiveness.

Our team decided to create a lesson plan for machine learning because we want to spread

the accessibility of technology to more individuals. With the frequency and abundance of

machine learning technologies emerging on the market, curiosity about what happens behind the

scenes with these technologies is also rising. When our team realized this, we decided that it

would be beneficial to create a small lesson plan describing code and the science behind it. To

many people who are unaware of machine learning, it may seem extremely difficult to learn. Our

team thinks that it can be simplified to the point where someone with intermediate Python skills

can not only understand machine learning models but also use them in their projects. With the

creation of this lesson plan, we hope to give more access and opportunities to those who do not

know where to start regarding machine learning.

 The hot topic in both the tech world and mainstream is AI. While we weren’t focused on

specifically AI, typical GPTs are built on machine learning models. Moreover, machine learning

is a rapidly growing field that has permeated almost all corners of global business. For example,

in retail, machine learning will continue to improve product personalization and ease data

analysis. In healthcare, machine learning will be used to improve diagnostics, helping to aid

treatment plans and prevent diseases before they happen. Finally, quality control and automation

will be changed forever in manufacturing from machine learning. These are just a few examples

of the implications machine learning has on the global economy but the proof is there. Knowing

machine learning will increasingly become an essential tool to have for any developer. This is

why we felt passionately that learning it should be accessible.

From our initial learnings and passion around the topic we devised this problem

statement:

It’s difficult for students not well versed in machine learning to digest ML code in a way that’s

easy and fun.

So…

How might we design a tool that helps students learn the basics of machine learning algorithms

in a simple, web-based, product?

Background

In the past few years we have seen more and more of a need for accessible machine

learning (ML) education, with numerous companies and resources starting to create platforms

that are built to benefit learners of various backgrounds and skill levels. Some of these websites

and services include Amazon Web Services (AWS) machine learning tutorials, Google

Developers, and Codecademy. While there may be a plethora of resources for people to learn
about ML, not all programs and lessons are easily understandable. Each of these platforms

utilizes different approaches in addressing the challenge of making ML concepts understandable

and engaging.

Amazon Web Services (AWS) offers a comprehensive collection of machine learning

lessons that are designed to simplify the process of building, deploying, and running machine

learning applications in the cloud. AWS provides a set of machine learning services and

supporting cloud infrastructure, attempting to put machine learning in the hands of developers,

data scientists, and expert practitioners. AWS's tutorials give people hands-on learning

experiences, allowing them to interact with real-world datasets and develop practical ML skills

using tools such as Amazon SageMaker. These tutorials and lessons would be beneficial to

someone who is looking to leverage and integrate ML solutions in their existing programming

projects. However, most people may find AWS's services to be quite complex and intimidating,

especially for someone who is new to computing in general.

Google developers is another example of a big tech company that has deployed a

program built to be a resource for existing programmers and data scientists. The popular online

learning program hosts a wide variety of lessons ranging from beginner tutorials to advanced

courses. Google developers utilizes libraries such as tensorflow and Google’s open source ML

framework, and covers topics from basic ML concepts to deep multilayer neural networks. While

this platform does provide people with the opportunity to either learn about ML or further their

knowledge in the field, it does not focus primarily on people who are new to programming. Due

to this, the instructions and tutorials might be too challenging and technical for the average

person who wants to learn about machine learning.

 Codeacademy is an online platform that offers a collection of tutorials and lessons in a

number of different programming languages. Dissimilar to the previous examples, Codeacademy

features sections that are specifically made for people who have never written a single line of

code before. In this website, each section for a language has lessons for different libraries and

concepts, making it much easier for someone to begin their programming career. In their

machine learning course, Codeacademy provides the user with an introduction to ML concepts

and an interactive coding lesson that explores different projects that use ML algorithms. By

going through a project and using the libraries and concepts being taught, someone has the

opportunity to start or improve their ML knowledge. Something that sets this website apart from

the others however, is that verified users can add their own lesson plans and explanations of a

library, not only creating an evolving learning experience and also a community within the

lesson.

Ultimately, Amazon Web Services, Google Developers, and Codeacademy are all

valuable resources for people to learn about machine learning algorithms whether that be

beginner concepts or advanced deep neural networks. Although they seem similar, each of these

platforms reaches a different target audience and employs a different approach to teaching ML.

In our project we aim to utilize some of these approaches such as interactive tutorials and the

ability to write your own algorithm while following along.

Objectives

To best reach our goal of making machine learning accessible, fun, and easy to learn for

anyone we needed to create objectives so that our success could be measured. In order to solve

our research question we narrowed in on three key pieces: conducting user research, creating the
algorithm, and implementing a high fidelity design to showcase our work. Out of these key

objectives grew four overarching goals that would help to govern our approach throughout the

project.

1. Make learning complex algorithms accessible.

2. Create a facial recognition algorithm that prioritizes diversity and eliminates bias.

3. Equip young developers with increasingly sought after skills.

4. Engage in a comprehensive design process to deliver a clean product.

We knew that if we met our goals we would have created a successful product we were

proud of. To get there though we needed to produce project management strategies which would

keep us on track. Ultimately as a team we decided on using a week by week timeline method.

This timeline had key milestones each week for all team members. The timeline also included

secondary tasks and meeting dates and times. While we weren’t always perfect on adhering to

this plan it was incredibly useful in managing our time. Ultimately we met every objective on

our list and hit all the goals we set out to achieve.

Methodology or Approach

We first began this project with the idea of presenting a live camera with facial

recognition software. We would train the dataset using the application “LabelImg” along with

using the library opencv2 with tensorflow to use the live camera. After working on this idea for a

couple of weeks, we realized we needed to pivot the project to something more basic. The first

reason we pivoted the project to a more simple model was due to teachability. The goal of our

project was to not only create a machine learning model, but also one that teaches the user. The
other reason we pivoted the project was due to technical limitations, setting up the model on

three different computers would be a momentous task.

We decided to settle on a Deep Neural Network with three celebrities for facial

recognition. We felt this simplicity of the project would display how a Neural Network operates

in python briefly and effectively. The use of a neural network in python also displayed content

our team previously learned in the machine learning class. This was essential because we wanted

to display what the information science department has taught our team. After creating the

model, we found that the model ranged from a 60%-65% accuracy on our tests for facial

recognition. We felt this displayed as a proof of concept, and were satisfied with this accuracy.

We felt satisfied because given the model and the amount of training data we gave it, it properly

displayed how a neural network actually works.

After creating the model and testing it, we then conducted a survey to gauge interest

about learning this concept to students our age. Grady conducted the survey along with gathering

respondents. Grady gathered around 30 respondents, which mostly displayed interests for

machine learning. This was important to our project because if we wanted to teach machine

learning, we needed to find individuals to teach. With our responses for our survey in, Kyle then

created an infographic displaying our results.

The survey was designed to inform our project structure and make sure our initial

thinking was on the right track. The survey was focused on confirming that our target audience

was interested in machine learning and would use a product like the proposed ML Launchpad. In

total, the survey consisted of 8 questions. We kept the number of questions small because we

only wanted to focus on crucial information so as to not lose the attention of survey takers. We

asked respondents their highest level of education, and if applicable, their area of study. This was
done to get a tiny bit of demographic information on respondents. We also asked respondents

about their knowledge, or lack thereof, on machine learning and facial recognition technologies

to get a baseline on people’s experience. Some other key questions asked were: Do you have

interest in familiarizing yourself or building up your machine learning skills? And Would you

use an online demo to learn a basic facial recognition algorithm? Ultimately the responses we

garnered confirmed that our concept was a go and that the tool had potential or real application.

After the team analyzed the results, Kyle produced an infographic with the significant findings.

We wanted to display our code and full scope of work on a high fidelity website to tie

everything together. To do this it was crucial to go through a comprehensive design process that

was similar to design strategies learned in the information science major. The first step of the

design process was to create user personas based on survey results and background research. The

personas identified were the beginner and intermediate coder. To best understand the behavior of

each persona, journey maps were created to feel out how each user may interact with the product.

Through this we learned that it would be extremely important to ensure the beginner coder could

be set up with a development interface on their personal computer. We felt that for both personas

that we needed to combine learning with ease of use so accessibility was upheld.

After personas and journey maps were complete the in depth design process began.

Prototyping began by creating low fidelity wireframes using Miro. The lo-fi prototype laid out

all web pages and also included ideation on the copy which would be used to populate the site.

The most important component of this prototype was designing the “learn” page. This page

needed to include a flow that walked the user through set up basics but also got them the code

and the dataset to run the algorithm. The lo-fi wireframe went through a round of user testing

with three beginner coders to ensure we were ready to move onto website design. In this user
testing round we paid close attention to how participants navigated the “learn” section. After

testing was complete edits were made to the Miro wireframe and high fidelity development

began.

The high fidelity prototype, which became the final design, was created using

Squarespace. Originally the team explored building the website from scratch but due to time

restrictions it was decided to use a traditional website development tool. Since the lo-fi

wireframes were well curated, website design consisted of mostly transferring information into a

visually pleasing design. To do this a style guide and brand kit were created for ML Launchpad

that included color palettes, fonts, logos, and secondary images which could be used to spruce up

the website. Once design was complete a final round of user testing took place to ensure our

design benchmarks had been met. View the final website design here.

Considerations and Limitations

The first limitation that we encountered was our own prior knowledge and experience

when it comes to creating machine learning algorithms. Since Kyle and Jack only did one

semester of machine learning, much of the time was spent going over notes and relearning

concepts from the previous semester. Along with this, the team was ambitious to use complex

and strenuous models. After much deliberation with different models, the team decided to stick

with a basic model that properly displayed how a neural network works. Another limitation that

the team encountered was the audience. Since the audience that was being targeted was

intermediate coders, the team needed to create the project accessible to coders while also

displaying a complex neural network. The consideration of the audiences prior coding

knowledge was integral when designing the website and code.

 Another limitation was bias in the algorithm, as facial recognition software is especially

prone to bias. Although the model only contained three celebrities, we needed those three

celebrities to be diverse so the model is trained on different faces. Along with this, we found that

there are a limited amount of celebrity headshots online. This limited amount of headshots for

celebrities made our model prone to overtraining, as there was less data than our team would

have liked for the model. We decided that each celebrity would have 10 headshots each. Seven of

these headshots per celebrity would be used to train the model, while the other three would be

used to test it.

Team Contributions

Jack:

The first contribution Jack made was working alongside Kyle with the aggregation of

data alongside the creation of the model. Jack first collected the celebrity headshots through

google. He then created a file path that could be used in the model. After creating the file path,

Jack then worked on the model and created working code. After this was completed, Jack

worked alongside Grady in the design process of the website and the lesson plan. After working

on the initial website design, Jack worked with Kyle in creating an informational introduction

video for the website.

Kyle:

Initial phases of ideation were completed by the group as a whole. After deciding on an

idea to move forward with, we split out the work. With grady being advanced in web

development and user experience frameworks, we assigned to him the development of our

website and user testing practices. On the other hand, Jack and Kyle were assigned with
completing our machine learning model in jupyter notebooks, and analyzing data collected by

Grady. Early in our process Grady created a survey which would help us get a sense of our target

audience's interest in our product. After receiving results from this survey, Kyle gathered the

information and displayed it in an infographic using Canva. Due to difficulties with collaboration

on code, a majority of the final code was executed by Jack. After the initial data collection, Jack

and Kyle each worked on their own models and then convened twice a week to talk about what

was completed. After seeing lots of errors in Kyles model, the group decided to go forward with

Jacks making it the main focus. Kyle and Jack then refined the model attempting to get a higher

accuracy score. Once the code was finished, Kyle and Jack filmed an informational tutorial video

for the website.

Grady:

Grady participated in initial phases of ideation with the team. After ideation was

complete, Grady began to craft a survey which would confirm if there was a need for the concept

or if a pivot was needed. The initial survey protocol was reviewed by the team and some items

revised. Once the questions were ready, Grady built the survey in Qualtrics and deployed it,

sending it to peers who fit the target demographic. Data on the survey was collected over a two

week period. Grady then worked alongside team members to analyze the data. Grady wasn’t

involved in a hands-on role with crafting the facial recognition algorithm. However, he did assist

in curating the datasets that would be used in the model. Most of Grady’s efforts were focused on

leading the design process. He created user personas and user journeys. He then created low

fidelity wireframes for the website design, ultimately turning these into a high fidelity prototype

and final design. Grady worked iteratively through the design process, conducting user testing
sessions along the way to ensure high functionality and quality visual aesthetic for the final

product.

Deliverables

Survey:

The ML Launchpad interest survey was the first major deliverable on our roadmap. The

survey was built in Qualtrics and consisted of 8 questions which sought to determine

respondent’s education experience, knowledge of facial recognition and machine learning

algorithms, and gauge if they would be interested in using a product like ML Launchpad. The

survey garnered nearly 30 responses from people across multiple education levels and with

varying levels of knowledge around ML and facial recognition. From the survey we were able to

determine that the majority of respondents would use a tool like ML Launchpad to learn or grow

skills in machine learning.

Website:

The final and live website is a culmination of the entire design process. The website was

built using Squarespace and can be found at www.mllaunchpad.com, a domain owned by the

team. The website consists of four pages. Firstly, the home page which includes entrances to

other pages, our mission statement, and information on what machine learning is and why it

matters. Then there’s the learn page. This page is where the code and set up information lives.

This is the most important page on the website. Next, we have the process page which holds

information on the project, our driving goals, and more learning information for users. Finally,

the team page gives information on each team member to build the credibility behind our

product.
Code:

For the code, we needed the model to display the process of identifying celebrity

headshots alongside showing the accuracy. Our goal was not to have the most accurate model,

but instead to display the actual process of creating a neural network. Alongside creating the

model and displaying the process, we wanted to add notes in the code that would help users have

a better understanding of what is going on in the background.

Conclusions and Future Work

Throughout the course of this semester, we learned many things not only about machine

learning but also about the different processes and frameworks that are taught in the Information

Science department. Although time was a limiting factor in this project, causing us to hit

roadblocks and make shifts in our timeline, we persevered through the times of doubt and

unknown leaving us with a project we are all proud of and excited about.

Looking ahead, we recognize the potential for further development and expansion of the

platform. Collaborating with more professors or individuals with expertise in machine learning

would keep us on the cutting edge of the industry, allowing us to provide the most popular and

commonly used models and libraries. In addition to these interviews, applying the facial

recognition model to other platforms and incorporating different algorithms would allow us to

provide a much wider variety of learning possibilities. For example, we imagine that by making

our lesson available for download in google colab, even the average normal person can engage

with our content.

Although people are familiar with facial recognition, machine learning encompasses a

large array of models that can perform many different tasks, which is why it would be beneficial
to include all or numerous models in the future. Many of the websites we looked at in the

competitive analysis section featured a multitude of lessons for all skill levels. This is something

we would like to incorporate in our own platform, and were unable to add due to time. This

would include introductory lessons for the average beginner to the advanced novice.

Lastly, we feel that there are always ways to refine the design process further, which

could lead to a more streamlined and user-friendly experience. Overall, the journey of creating

ML Launchpad has been both challenging and rewarding, and we look forward to exploring

future opportunities for growth and innovation in the field of machine learning education.