Machine Learning for Photographers: Image Tracking

Basic idea: as a photographer, if you’re shooting street photography (or photographing anything that is moving), you are tracking the movement of the subject. The movement of the subject moving towards you, away from you, or in different directions. But how can we apply this machine learning concept of ‘image tracking’ to our photography? Some ideas:

Learn to see like a machine

The first time I saw this animation from the slides from the UC Berkeley CS 280 (Computer Vision) course, I was blown away (also see the more extended, 2015 course here).

Basically I see a brave new world in photography in which your phone camera will have “AI (artificial intelligence) assistants” — that will assist or coach you in composing better photos.

For example, imagine this:

You see an interesting scene, and you can (in real time) see your subject abstracted into a 3D model, and the distracting background is photoshopped out.

This will allow you to better figure out how to pose your subject, from what angle to shoot, and when to take the photo (timing).


Machine Learning / Deep Learning Technologies

machine learning

Right now in Silicon Valley, AI, machine learning, deep learning is the hot stuff. Basically the question that a lot of researchers are trying to address is:

How can we best leverage technology to live better lives?

Unfortunately a lot of tech folks are nerds who are more obsessed with trying to create a ‘sentinent’ god-like AI that will make humans superfluous. Fortunately my buddy Elon Musk (and others) are pushing against the dwarfing of humans, by creating organizations such as OPEN AI to figure out how to best use artificial intelligence in a meaningful way that actually helps humans.


How can machine learning/ai help me as a photographer?

Anyways for myself as a photographer, I’m interested how we can best use/leverage ai, machine learning, deep learning, etc to improve ourselves as photographers. To make better compositions and images. Ultimately the point is to Crete a ‘centaur’ — a photographer who uses artificial intelligence to assist them in making better photographs. The point isn’t to make photographers irrelevant.

And one of the pillar-concepts which are helpful is ‘image tracking‘.


The future is already here

The cool thing is that we already have technology in our phones and digital cameras which have image-tracking built in. For example when I shoot in “intelligent auto” mode on my Lumix G9, I get these little yellow ‘tracking boxes’ which assist me with autofocus, and also to just track my subject.

As a photographer, you can use this technology to track moving objects, cars, or trains:

Or perhaps track a single object or subject (like a single fish) when there is a school of fish:

Or using image-tracking to help you automatically focus on a tiger as it is approaching you; either if you’re shooting a video, or just still photos:

Anyways once again– the point of AI is to help us be better photographers. The point is for technology to help SIMPLIFY our lives as photographers, so we can focus on what is truly important to us:

Making meaningful images.


Brave new world of image detection

So anyways, there is a bevy of technologies. Let me cover some:

1. Keypoint Detector

The Keypoint Detector: use machine learning to automatically categorize our joints to categorize parts of our body:

For example, if I upload to this online demo this picture of this lady in pink, you can see how it automatically detects her joints:

Animated GIF:


2. Face Tracker

The face tracker: technology to assist us to track faces, and also with auto-focusing in live video:


3. Pose based Video Retrieval

Leverage technologies to get new and novel ideas for posing your subjects, by looking at videos from the past, to help spark random and novel ideas:


Conclusion

Ultimately, I want us to make technology our slave; not for us to be slave to our technologies. But unfortunately in today’s brave new world, we are all slaves to our phones, notifications, emails, social media, etc. But what if we took the power back in our own hands?

I encourage you to think about your personal use of technology in your everyday life, and think about how you can best leverage the good parts of technology, and subtract the superfluous and distracting.

BE BOLD,
ERIC


Machine Learning x Photography

Brave new world of photography:

<

ul>

  • Composition and Machine Learning: Bounding Boxes for Photographers
  • How Photographers Can See Like Machines
  • Why AI (Artificial Intelligence) is Good for Photographers!