What AI Is Good For
Posted on March 3, 2023 Posted by John Scalzi 19 Comments
At the moment we’re all sort of freaking out about the uses of “AI” in text and art, because even as overhyped as it is, it’s still exhibiting some genuinely transformative skills it didn’t have even a year or two ago, and those skills will only get more refined as we go along. People should be curious, and concerned, about where this all goes and how it will affect them, even as they (hopefully) look beyond the hype, good and bad, about it.
With that said, I will say that one area where I am frequently delighted about the advances of “AI” and machine learning is in the realm of photography, where the current level of tools is working to make my photography better. Not by slathering TikTok/Snapchat-like filters on people, but by working to mitigate and minimize hardware issues with my camera.
For example, the picture above of Spice, which I took with my Nikon. I took the photo in low, natural light without a flash, and with my camera set to “auto.” The picture out of the camera has a lot of sensor noise to it, i.e., the speckles especially noticeable in solid colors that is often called “grain.” Grain is not a bad thing in itself — photographers often choose to have it for aesthetic effect — but when you don’t want it, it’s in the way. This picture of Spice does not benefit from it. There are historically a number of ways to mitigate grain in a photo, but if poorly used they will end up making a photo look plastic-y and featureless, which is the say the solution is just as bad as the problem.
But now there’s a new generation of photo plugins that use machine learning and “AI” to wipe out grain and have it look… pretty darn natural, by which I mean, like you took the photo with the correct amount of light to avoid sensor noise. It doesn’t add anything to the photo, and it doesn’t call attention to itself, it just helps make the photo look closer to what you hoped it would be when you took it.
Which is great! It means as a photographer, I have a little more flexibility in taking shots that will eventually look good, and it also means that I can extend the use of the camera/lens/etc that I use for photos before having to upgrade, which is awesome, because cameras and especially lenses aren’t cheap. Like any tool, these plugins can be misused and abused, and you can still go full plastic if you make an effort. Moreover some pictures still can’t be salvaged no matter how hard you try, even with plugins that also upscale and color correct and get rid of JPEG artifacts and so on. But in general, these sorts of tools make life better.
Which is the upside of “AI” and machine learning tools: Not to replace human creativity and effort, but to work with it and make it better, i.e., just like any other tool. I like that and want more of it across all my creative endeavors. I like tools that let me do more, not do it for me.
— JS
The plugin in question here:
https://www.topazlabs.com/topaz-photo-ai
I was wondering which plug-in you were using. I’ve used that and the sharpening tool from the same source. I haven’t seen a need yet for the upscaler – even with some cropping I wind up with a large enough picture for anything I use it for.
Seems like a fine tool. But does it in fact rely on machine learning or anything of that sort, or is it just a clever smoothing algorithm? Both seem plausible.
I’d guess that you would need machine learning past a certain point of “unsalvageability” by traditional methods, but are we there yet?
No idea, myself, but maybe you or one of your commenters knows more about what’s under the hood.
Extremely lovely picture of Spice, who I feel does not get enough attention here. Oh, and the words about photography and AI? They were lovely as well. :)
Having done precisely no research, I wonder how AI audio filters are doing these days? De-essing and noise reduction seem like great candidates.
Topaz Photo AI is wonderful! I use the standalone app and am constantly dazzled by what it does to my collection of images, many of which are scans of 35mm Ektachrome slides. They are good scans, made carefully a decade ago with a Nikon scanner, but limited in resolution, typically 4000 x 3000 pixels. Topaz takes one of those and doubles its apparent resolution, reduces the noise, and brings up amazing detail.
I compare the original and the enhanced versions, and yes, I have to admit that the AI invented some of those pixels, just made them up of whole cloth. But the invented pixels are the right pixels, nothing visibly artificial or over-sharpened, no perceptible artifacts. It just magically finds the photo I took, hiding in the photo I have.
I think my biggest take away from this post is what this blog needs is more pictures of cute animals.
Cheeses! When I last took a photography class (in Oberlin’s art museum at college) a good part of the lerning was how to pick the exactly right gray inks to fix any little spot or line of dust that had made it past meticulous cleaning of the (black and white) negative. Our teacher told us that his own final exam was being given a photograph of an outdoor fountain’s spray and being told to take out the water.
And now you just press a button. The future is dazzling.
I’ve been using Luminar Neo as an ai photo editor. It’s really nice for someone like me who doesn’t want to spend the time and money for Photoshop. Has an auto dust remover that’s really good, and a power line remover that’s ok. Lots of other tools as well. Made by a company in Kyiv, but the UI is in English.
Is it possible to show what the photo of Spice looked like prior to the use of the AI filter, for comparison?
It’s funny, I’ve been comparing AI audio innovation to smart-phone photo editing because more people are familiar with the latter. Audio editing & processing has advanced remarkably in the last few years, in large part because of AI. Software’s ability to distinguish a vocal from ambient noise has (in particular on Waves Clarity) reached the point where I could record you in a bus terminal and end up with a good dry vocal, entirely by turning a knob scaled from “ambient” to “vocal.” Related & more specific plugins remove room reverb, breaths between sentences, etc.
AI on mastering software such as Izotope’s Ozone 10 can use psychoacoustics to place instruments in the audio field, widen them in the stereo space, carve out space for the vocal from the instrument spectrum, and quite a lot more.
I very recently upgraded my laptop and OS (thanks to my awesome listeners!) so that I could take advantage of this, because things are moving fast & I felt I was getting left behind. It’s pretty remarkable.
::Which is the upside of “AI” and machine learning tools: Not to replace human creativity and effort, but to work with it and make it better, i.e., just like any other tool. ::
Exactly! I’ve been using Topaz Labs’ Video Enhance AI for the past year to do the same thing to old standard-def videos, by having them enhance them to HD or even UHD resolution. It’s not perfect, but it’s a lot better than just increasing the resolution of the video like I was blowing up a picture….
Your last sentence says it all, for me. I prefer a tool to enhance my work, not do my work.
Bill: My experience is that photos at your monitor’s resolution (for displaying web pages) don’t show enough detail to strongly show the noise or the improvements. Of course, monitors vary.
That’s also to say, most image work I’ve done as a designer in the last decade or more has been intended for the web, and I’ve gotten away with so much.
And its 3-Laws safe!
My iPhone (and presumably most flagship smartphones) uses AI to pick out humans in photographs, and make them look well lit even if they are backlit in the photo. As the photo loads in the viewer, you briefly see the uncorrected photo, and you can watch their unlit faces look as though they’re lit by studio lights. But not ridiculously so. As you said, it turns the picture I took into the picture I wanted to take, and with no extra effort on my part.
It’s so good at recognizing people that you can click on a person in a photograph, and it will select just that person. Copy and paste that selection into a new image, and you get the person with no background.
Every year, camera phones get features and effects that you previously could only do with expensive pro or semi-pro equipment. Purists will say they still don’t match the quality of the pro equipment, and maybe so. But each year the cameraphones get better at creating images that make me happy I took them.
Topaz Labs products have impressed me for a few years now. I enjoy using their Video Enhance AI for cleaning up captures of old VHS recordings and make them look a bit better at higher resolutions. For videos that were captured on older digital cameras or DVDs, the results can be downright magical and look like a HD re-scan from original film.
“I like tools that let me do more, not do it for me” and yet those same tools allow many people to simply set it to “auto” and let the tool do all the work.
This applies to writing. Some people may use AI to write for them, while others may use AI to enhance their writing (btw if you have any ideas on how AI could enhance your writing, please share).
One may see it as AI raising the bar on everything it touches. Although since AI is trained on people’s output, it’s a never ending arms race of creativity that either side can win.
I started playing with ChatGTP last week. While I was at a truck stop in Lordsburg, I had an idea for a question: Can you translate French to Sindarin?
Sindarin is the main language of the elves in Lord of the Rings. I run a band of 15 characters in Lord of the Rings Online. And I’ve been wanting to do that translation for the song by Electric Light Orchestra, Hold On Tight (To Your Dreams).
And ChatGPT provided!
I hadn’t gotten it done because it just wasn’t a priority, too many other songs to make or revise or lyrics to rewrite. But now this is done. I do need to double-check the translation against a Sindarin dictionary (I think it’s elfdict.com), but there’s only three lines, so that won’t be too bad.