Video to Audio to Transcript to Translation via AI

The well-known speaker Thomas MacEntee recently gave a presentation to my local genealogy club about how to use AI. During the Q&A period, I asked him whether AI could translate and transcribe foreign language audio for free. His answer was a definite yes. So this week I began to experiment.

I have a 1998 video interview of my mother's first cousin, Viola, speaking emotionally about her experiences as a Holocaust survivor and her early years with her family. The interview was conducted in Russian in Israel through the USC Shoah Foundation. A decade ago, a friend who knows Russian kindly translated the gist of this video interview. Now I wanted to see what AI could do for me, for free, in better understanding family history comments that Viola made early in the interview. 

Process: video audio to digital audio to transcript

First, I popped the DVD into my player and as soon as the interview began, I started recording a voice memo on my iPhone. My first audio recording was 11 minutes long. This was important because Thomas said that without a paid AI account, it's better to keep projects shorter and simpler to get things done.

Next, I had to change the m4a recording to mp3 format, which I did with a free online converter (I used CloudConvert but there are other sites out there). 

I tried uploading the mp3 to ChatGPT (free version) and asked for a transcription and translation from the Russian. But the AI responded: "It looks like I can’t run Whisper (speech-to-text) directly in this environment, so I can’t transcribe the MP3 automatically here."

So I uploaded the mp3 recording in Russian to TurboScribe (one of many sites that do this) and I asked for a free transcription. I chose the best quality/accuracy and within minutes, I downloaded the written output as a pdf.

Formatted translation from the Russian

Finally, I took the pdf of the Russian transcription and uploaded it to ChaptGPT, explaining a bit about this being an interview. I asked for this transcript to be translated into English and formatted as interviewer and interviewee. 

The AI had no difficulty distinguishing between the words of the person asking the questions and the person answering. It did ask whether I wanted a summary or a complete transcription (I wanted everything). Also it asked whether I wanted some original terminology left as is, with translation in brackets (yes).

ChatGPT finished that initial translation and asked for me to upload more so it could create a single, seamless document. So I went back and recorded 6 more minutes, going through the audio to digital audio to mp3 conversion rigamarole, next getting the free transcription, and then uploading the pdf from this second segment to ChatGPT.

This time, I named Viola as the interviewee and the AI showed her name in front of all of her responses. In the blink of two eyes, the answer showed me both segments compiled into one seamless interview about Viola's mother, father, and grandparents and their life before World War II. The top of page one is shown here.

Output: Formatted to my specs

Chapt GPT preserved some of the original terminology (see image at top, look for the word for tavern) and some of the less distinct words were picked up and translated, too. 

No cut and paste for me. I just asked for a .docx Word document, which was quickly created for easy and free download. The output is in complete sentences, with proper punctuation, a smooth read. I added a note that ChatGPT had created the document and the date. Done!

Use with caution

ChatGPT warns that it can make mistakes (see image here). I also asked it to please delete the file at the end, after I had finished my download.

Given how many steps were needed to go from video interview to final document, there are multiple opportunities for mistakes/omissions/typos to creep in. Very likely there are some nuances that got lost along the way but in the end, I believe this was a successful experiment. Thank you to Thomas MacEntee for the encouragement!

Getting Context from Ancestry's LifeStory with AI Insights

Still in beta testing, AncestryAI is an option to provide historical and social context for some elements of the LifeStory feature on an ancestor's profile page.

Above, the early section of the LifeStory for my great uncle Abraham Berk (1877-1962). This takes the form of a timeline including family events such as his birth and the birth of his siblings, followed by his residential locations in chronological order, marriage, immigration, birth of children, occupation, and actually all events on his profile page.

Next to some of these LifeStory items is an option to Ask AncestryAI about a time and place in the ancestor's life. The AI summary helps add context to the ancestor's background. As shown above, here the AI is explaining what Lithuania was like in 1877, the year of Abraham Berk's birth. 

Other options to learn more about a time and place include what would someone wear to a wedding then, what mourning customs existed then, what was entertainment like then, what local dishes were popular then (see below, about Lancashire's fave foods in 1901).

While much of the AI-generated info is general, it still is a helpful starting point for understanding an ancestor's life and can give clues to fleshing out a bite-sized bio with context. Worth checking out if you have this feature in beta at Ancestry.

Note: Yesterday's blog post looked at FamilySearch's Brief Life History feature. Tomorrow's blog post will look at MyHeritage's AI Biographer feature.

Ancestry's Ask AI Feature

This week, when looking at a German language Hamburg passenger manifest for one of my ancestors, I noticed that Ancestry was offering me the opportunity to explore this document using its Ask AncestryAI feature. Of course I took a look! I haven't tried any Ancestry AI feature before this.

Above, part of the screen, with the AI interpretation on the right and the transcribed record (plus the original image) on the left. The AI seems to have used the info from the record detail and constructed a narrative that began "Bela Roth, a Hungarian male, departed from N. Bereg, Hungary, at the age of 42 on October 17, 1907..." 

The AI named the ship, the ship's ownership, and said he was a merchant (true) and was accompanied by six other household members (true) including his mother (not true) sons (true), and father (not true). The AI concluded by citing the source of this record.

In actuality, Bela was accompanied by his wife and sons, and he named his mother as the nearest relative in place he left. Bela, himself, was the father of the sons.

Deciphering a handwritten manifest is a challenge, and when I don't know the language, it's an even bigger challenge. Here, I had to compare what the AI said to the actual record AND to my family tree to understand what was true and what was not true. Also, the AI had no way of knowing that N. Bereg = Nagy Bereg. But I knew the full place name from prior research. I tried the AI feature on another German-launguage Hamburg passenger manifest, with similarly mixed results. 

I can imagine situations where the AI assist would be helpful. Still, IMHO, there's no substitute for understanding a document's purpose, timing, format, content, and trying to decipher it independently from what the transcription says and what any AI assistance says. I'll continue to test this feature, hoping to learn a few new details--that I'll confirm for myself.

UPDATE: This AI feature is available for a variety of documents. Here's what it told me about a man's WWII draft registration card. I clicked the thumbs down on this description--the man in question did not serve, and the narrative is rather flowery without much substance, unfortunately. Oh, and let me quote the disclaimer from the bottom of the screen: This feature is powered by an AI language model using only information from this record. Responses may be inaccurate. 

Try this feature if you can, and see what happens!