A look at TwitterFone's accuracy (and we're giving away beta keys!)

With TwitterFone rolling out local numbers for a bunch of countries yesterday, the service seems to be quickly moving toward the end of its private beta phase. I figured it was about time to put TwitterFone’s voice-to-text tweet abilities to the test.

To test it out, I put TwitterFone through a handful of situations it might be used in regularly and compared what I said to what TwitterFone transcribed. The tests definitely weren’t scientific by any means, but they still go to show whether or not TwitterFone’s voice transcription services are up to par for day to day use. I ran each test in succession without peeking at the results, so as to keep myself from subconsciously adapting the way I spoke to it between tests. Differences between what I said and what was transcribed are highlighted in red.

(Oh, and if anyone wants to know when I’m doing my laundry or eating a sandwich, feel free to follow me on Twitter.)

Test #1: Quiet environment, speaking informally:

What I said: “This is just a test of TwitterFone, talking to it like I would a normal person. We’ll see how the voice recognition…” (Lost cell signal, call terminated. That last bit would have been “… software works”.)

What TwitterFone transcribed: “This is just a test of Twitterfone, talking to it like a normal person. We’ll see how the voice recognition… http://tinyurl.com/6lpml9″

Result: Not bad. I tend to speak pretty fast, so it doesn’t surprise me that it didn’t catch the “I would” part. Regardless, it ended up meaning about the same as what I intended.

Test #2: Quiet environment, speaking at a slower pace:

What I said: “This is the second test of TwitterFone. Enunciating a little bit for the sake of the voice recognition software, so we’ll see how that works.”

What TwitterFone transcribed: “This is the second test of Twitterfone. Enunciating a little bit for the sake of the voice recognition softwar… http://tinyurl.com/6z7w45″

Result: Dead on. It cut me off a bit early, but the provided TinyURL lets people hear the rest..

Test #3: Music playing in the background at around the “Wait, someone’s calling, turn it down” level:

What I said: “This is just testing Twitterfone with some low level music in the background. This is testing background noise.”

What TwitterFone transcribed: “This is just testing Twitterfone with some low-level music in the background. This is testing background noise. http://tinyurl.com/57y24l”

Result: Perfect, even going as far as throwing a hyphen into “low-level” for me. Oddly, I can’t get the audio playback to work for me on this one.

Test #4: Music playing in the background at around the “My ears, they are bleeding” level:

What I said: “While I don’t expect this to work, this is a test of Twitterfone with music playing behind me really loud.”

What TwitterFone transcribed: “Well, I don’t expect it to work, (?) Twitterfone, music’s playing (?) really loud. http://tinyurl.com/6z83jz”

Result: As stated, I didn’t expect this to work at all. Loud music, for obvious reasons, screws things up. Don’t try to tweet via your phone during a rock concert.

Test #5: Fancy words:

What I said: “The epitome of Copernican astronomy ”

What TwitterFone transcribed: “The epitome of Copernican astronomy. “

Anyone looking to have intellectual conversations via tweet is in luck – the speech recognition system seems to understand all that crazy city folk talk (or atleast, the backup human transcribers do).

Bonus Test: Terrible, terrible rapping of the Fresh Prince theme.

What I said: “This is a story all about how my life got flipped, turned upside down”

What TwitterFone transcribed: “This is a story, all about how my life got slipped turned upside down. http://tinyurl.com/6b5z2u”

Result: On one hand, it totally did sound like I said “slipped”. On the other, this is Fresh Prince; all transcription services should be trained in the ways of big willie from day one.

Post-Test Results:

As a transcription service, Twitterfone maintains a surprising level of accuracy, unless you put it in ridiculous situations where it would be more surprising if it could understand you. I imagine it’d be far more useful for those stuck with T9 input than it would be for those with QWERTY, as on QWERTY it’s just as easy to punch out a quick SMS and fire it over to Twitter. With T9 users in mind, I think it’d be great to see this service wrapped into Twitter’s official offerings, with an embedded play button replacing the TinyURL link.

After each tweet you record, the automated voice will say (in a voice suitable for just about any 1-900 number,) “Thats a good tweet“. At first it kind of creeped me out. Eventually, I began wishing someone would do that after everything I did. “That’s a good post.” or “That’s a good odor you’ve got after not showering for two days.” It just makes you feel good about yourself.

Beta key giveaway:

While they’re trying to keep the TwitterFone beta pretty tight, we’ve got a handful of beta keys to give away to our readers. Just leave a comment on this story, and we’ll pick five winners at random.