Dev90X: Zero to App in 90 Days

By Jack Cotton-BrownTechnology

Dev90X: Zero to App in 90 Days episodes

September 25, 2021Day 50: Handstand Vision V2 has been trained! Working with... CreateML, Quicktime, Instagram.
Discussion:
The limitations of Apple Vision
Strange scenes

Occluded body parts

Can’t find the face

Thinks it’s a shadow

Progress Update:
Handstand Vision V2 has been trained, and is an improvement on V1
Better recognition of entries

Better recognition of exits

Less confused about other types of inversions

Learned more of the limitations of Apple Vision

dev90x.com
12 min
September 24, 2021Day 49: 1 + 1 = 3. Working with... Instagram, raw videos, video editing.
Discussion:
Having another person involved:
Doubles the size of the accessible network

Smooths out the workflow

Holds more consistent momentum

Brings in a division of labour

Increases the team skill-pool

Increases quality of brainstorming

Enables reflective feedback

It also complicates a few things:
Division of value and equity

Progress Update:
Purchased the Apple Developer program for $150AUD

Downloaded more videos

Edited a lot of videos
Probably around 80 more clips

Did some logic verification on some videos

Created a list of ideal data requirements for my own reference to potentially teach others how to do this.

Built out more folder structure for the data.
dev90x.com
17 min
September 23, 2021Day 48: This project is becoming more serious by the day. Working with... Instagram, CreateML, Logic Design, Brand Design.
Discussion:
Portfolio project VS serious app project.
Progress Update:
Downloaded more data
Handstand walking

Edge case data - scenes with mirror reflections.

Sunset background, silhouette foreground.

Scenes with crowds of people

Designing the logic system
1D, 2D, 3D

1 second delay, 2 second delay, 3 second delay.

More comparisons on video compression and vision quality.
The drop-off point seems to be after 95% compressed. Marginal gains from 95-90% compression. Good performance on less than 90% compressed.

Talking branding with Harry and Dylan

Redesigning the folder structure for the training data

dev90x.com
28 min
September 22, 2021Day 47: Zooming out to the bigger picture again. Is this idea even viable? Working with... Adwords, Google Trends, Lossy Compression, Instagram & Harry.
Progress Update:
1 hour business call with Harry
Discussed target market

Discussed community targeting strategy

Talked about the initial app features

Identified some issues

Talked about cheating and the impact of that on the game experience for others

Mapped out some of the logic for the Handstand Judge
Realised that every logic sequence should begin with “other” as the starting pose, and end with “other” as the final finishing pose.

Downloaded more data from Instagram
40 more videos to edit and sort now

Continued to build the model stress test dataset to see how it performs on the toughest edge cases.

Did some trend analysis for handstands in google trends to find the most popular search term. (Which is Handstand (Yoga Pose))
Organic Google search traffic is definitely not a viable marketing strategy

Did a compression test to see how much compression reduced the accuracy of Apple’s Vision - surprising results.

Tested out the competition properly - handstand quest app test.

Realised there has never really been an easy way to track handstands until handstander.
dev90x.com
23 min
September 21, 2021Day 46: Training the AI feels like progress. Working with... Instagram, Create ML.
Discussion:
Getting back into training the AI - takes some time to get back into the flow of it.

Re-designing the training data set
Creating a collection of edge test data - random and strange videos

Do I put all things that are not handstands into the “other” pose - even headstands?

Pole-dancing videos give really interesting 3D postures where the camera is static, and the person is moving around.

Dancing videos give a great dynamic overview of movement patterns to test the model against.
Progress Update:
120 training cycles yielded 91/70.

240 training cycles yielded 92/73 (+1%, +3% accuracy)
dev90x.com
12 min
September 20, 2021Day 45: I’m getting help! Working with... Harry!
Discussion
First time experimenting with a work trade

The focus was mostly on Handstander
Progress Update
Testing the app on an iPhone X for the first time.
Running at a lower framerate but still seems to work at getting predictions.

iPhone X is probably the lowest that can be supported

Created an Instagram for Handstander

New Community group chat for collecting data in Telegram

Data collection strategy combined with alpha release strategy.

Started work on branding and aesthetic design
dev90x.com
14 min
September 19, 2021Day 44: It’s time for rapid iteration! Working with... Xcode, Guess My Handstand, App Design.
Discussion:
Now that I have a much better codebase to work from, it is enabling me to iterate faster on the core design and features.

The main priority is to improve the performance of the predictions so that it’s getting it correct every time.
Progress Update:
Changed the confidence threshold for the pose predictions from 60% to 35% so now I can see the feedback of all the low confidence predictions as well.

Realised if I am to design a prediction smoothing and logic system, I will need to continue to include many different pose classes to be trained into the model - which means maintaining correct volumes of training data for each class as opposed to throwing everything into an “Other” class and just keeping handstands as the primary.
I will experiment with this now to see how it works.

dev90x.com
18 min
September 18, 2021Day 43: Strategy assessment! What is the minimum effective amount I need to learn?
Discussion:
Strategy: just take tutorials and customise them to my needs.
Problem: Not a lot of tutorials available to suit my specific need.

Problem: Most tutorials are for completely different apps.

Problem: Most tutorials focus on learning UI and views - whereas my core app does not need this yet.

Problem: Vision framework applications (from what I can see so far) use a lot of complex structures of the language.

Strategy: take bits and pieces from example apps and open source apps and stitch them together.
Problem: Requires understanding the code

Which requires understanding pretty much the entire project.

It takes a long time to read and understand another project for me.

Strategy: Hire someone else to build the entire thing for me.
Requires an iOS specialist - not a flutter developer. Not as common or easy to find.

Requires a step back to project management and managing someone else doing the work.

Requires capital - I could probably do it with under 5k, but that’s more than I can currently afford.

Doesn’t result in as much learning for myself.

Cannot be a portfolio piece to showcase my development ability.

Strategy: build it all from scratch myself.
Requires that I gain swift fluency as I will be implementing my own custom design.

Most people take around 6 months to learn iOS app development from scratch. And this might even be a difficult project for someone who’s spent 6 months already.

At this rate, I will run out of time.

Strategy: hybrid approach.
Learn swift

Copy as much as I can from other projects

What is the minimum amount that I need to learn?
Understand all the basics of Swift

Understand object oriented programming
How classes behave

How structs behave

Understand project architecture and design patterns

dev90x.com
26 min
September 17, 2021Day 42: Progress happens in small steps, and your project needs to be a “F*ck Yeah!”. Working with... Xcode, YouTube, Google.
Discussion:
I think the project has to be a fuck yeah for you to be pumped up about the small wins, because the small wins can be monumentally difficult sometimes.
Progress update:
Good feedback from my instagram posts - all positive so far.

People are wowed by the vision skeletons - that seems to be the thing that attracts attention to the app

Some people want me to build features to help with yoga

I realised that the camera is a necessity - the user has to be able to see that they are in the scene, and being captured properly. I was thinking to remove the camera video feed view to save on battery, but I don’t think that will work.

Anxious about my crappy code turning into a mess
Life update:
Heading to the mountains for a day!
dev90x.com
11 min
September 16, 2021Day 41: Handstand Vision is plugged in!! Working with... Xcode, "Guess My Handstand", Real world testing!
Progress Update:
Handstand Vision is plugged in!!

The sample videos are on Instagram, and the first round of reactions are coming in.

Learned the correct way to refactor class names. Refactor and rename is the way to go in Xcode.
Next Steps:
Re-train the ML Model to include more, and better data.

Implement a basic counting system.

Implement audio feedback

Get some beta testers
dev90x.com
17 min

About Dev90X: Zero to App in 90 Days

From the publisher's feed

Join me on my journey to ship an app in 90 days from near-zero developer experience.

Share Dev90X: Zero to App in 90 Days

Sign up to save your podcasts

Dev90X: Zero to App in 90 Days

Dev90X: Zero to App in 90 Days episodes

About Dev90X: Zero to App in 90 Days