Twitter’s Killer App: Real-Time Search

Real-time search. Google doesn’t have it. Neither does ChatGPT. Microsoft, Amazon, Apple, and Facebook probably wish they had it, but they don’t. Twitter, Silicon Valley’s favorite shitshow, is the only one with this power. And for some inexplicable reason, it’s not doing anything with it.

Twitter has a tremendous amount of live, real-time data about what’s happening where. This makes Twitter unique as a service—it encourages its users to create in-and-of-the-moment content. They share what they’re seeing, thinking, feeling, and experiencing now—for example, millions of people tweeting during the World Cup final, thousands posting about a Taylor Swift concert, or hundreds discussing an emergency in their town. Something happens, and seconds later, people turn to Twitter.

In contrast, Google (and search’s other new darlings ChatGPT/Bing) are moving at a snail’s pace. Google’s web crawlers can take anywhere from three to 30 days to cycle the internet fully. Its most “of the moment” results usually pull from news outlets, which themselves may take two to three hours to cover a significant event. And plenty of real-time topics you may want to search for will never warrant news coverage. “Why is the highway near me closed right now?” is a perfect example.

ChatGPT only accesses data from 2021 and earlier, according to OpenAI. It has no information about events taking place right now. It can brilliantly analyze any existing data you present to it, but it can’t use the internet for updates or developments (yet). AI like GPT-4 has a role to play in real-time search, but these computing systems need someone—or something—to feed it real-time data. Enter Twitter.

The data that Twitter is sitting on is truly unique. If it can figure out how to interpret it and serve it to users via search, it could be a game-changer. And yet Twitter search, even the advanced version, has barely changed in several years. It’s complicated, not particularly user-friendly, and not tremendously helpful unless you want to find a specific tweet from a specific account or look at a bunch of tweets from a lot of different people, many of which are either not relevant to your search or not reliable at all. It doesn’t help that they’re presented in no particular order.

Twitter Advanced Search has barely changed since its introduction in 2018. Its function is limited (you can’t search by tweet location, for example).

Twitter search could be so much better. In fact, it could be the best, most valuable part of Twitter. Right now it’s a purely functional feature that has received minor updates in 2012 and 2018. But, what if Twitter search did more than just help you quickly and easily find relevant tweets? What if it could analyze those tweets and provide an actual answer to the sorts of questions that can’t be answered by anyone else on the internet right now?

How it would work

The idea for Twitter’s real-time search originally came to me when I lived in the Flatiron neighborhood in Manhattan. It was September 2016. I had just walked into my apartment when my wife, standing by the window holding our dog, turned to me and said, “I think something bad just happened.”

She heard a loud bang and immediately ran to the window to see what was taking place. I was in the elevator, so I hadn’t heard anything. I looked outside to the famous line at Shake Shack right across the street from our building. Everyone seemed to be living their lives and enjoying their burgers. All seemed well in Madison Square Park until seven or eight cop cars raced down 23rd Street. Even that didn’t seem to faze those waiting in line. 

I searched Google and found no information. I checked my local news and saw nothing. Finally, I went to Twitter and searched for “explosion + flatiron” and instantly came across several tweets about a bomb going off just a few blocks away from my building. This was the 2016 Chelsea Bombing, and in retrospect, a bunch of people eating at Shake Shack while a bomb went off may be the most New York City thing I’ve ever witnessed.

It took nearly two hours before major news outlets were saying anything about the bombing. Twitter was the only place I could go to learn more about what was happening near me. As helpful as Twitter search was then, it still served me tweets about “an explosion of new retail stores in Flatiron” and whether people’s laundry gear could spontaneously combust. Not exactly what I was looking for.

How would a better, more intelligent, real-time Twitter search work?

Mockup: The mockups above and throughout the rest of this article illustrate how real-time search on Twitter should look and function. 

Almost all tweets contain the same valuable data: the text and media content of the tweet itself, who tweeted it, when they did so, and the current location of that author. Twitter also knows the type of device used to tweet, but this is less relevant for our purposes.

If I had access to real-time search on Twitter during the Chelsea bombing, I would’ve searched something like, “Was there an explosion near me?” Let’s break that query down to explore how Twitter could quickly parse tweet data to give me an accurate answer.

“Was there…”

I’m asking if something took place in the past. Since I’m not specifying a particular time (like “yesterday” or “a week ago”), then the system should assume I’m asking about something in the very recent past (last hour or two). My query should look for tweets that hold data (text or media) that can confirm or deny that a specific event occurred recently.

“…an explosion…”

The specific event, in this case, is an explosion. Twitter will begin to look for keywords like explosion, bomb, blast, etc. It could also evaluate whether photos or videos related to those keywords are present. This is where AI could play a big role. For real-time search to be possible, Twitter would need to understand the content and the context of a tweet. While much of that can be derived from the text within the tweet, more advanced AI capabilities like computer vision would also be able to study images and videos. 

Knowing that an explosion took place within the recent past is not enough, though. Sadly, explosions are taking place and being tweeted about frequently all over the world. So, now we have to narrow the scope of this query.

“…near me?”

Twitter’s real-time search superpower is its ability to combine live data with location. The Twitter app knows where I’m located and can limit the tweets it evaluates to just those sent within my geographical area. It could do the same thing if I specified a different location. For example: “Is the French Quarter in New Orleans currently being flooded?”

Mockup: Through the use of AI, real-time search on Twitter could analyze and interpret those relevant tweets in order to provide an actual answer to a user’s question.

Let’s apply this same process for less catastrophic queries:

“How long is the line to vote at my local polling station?”

In this example, Twitter can cross-reference your home location with publicly available polling places to understand your query. Then, it will look for text and media related to the length of the line reported at that location. It may not be able to give an exact estimate, but it would be able to surface relevant tweets containing first-hand accounts and images. I recently did this to determine the length of the line for voting in last year’s Brazilian national election.

“Has the Taylor Swift concert started yet?”

This type of query is a particularly easy and useful one to answer. The start times printed on concert tickets are pointless (is it when doors open? Is it when the concert begins? Is that when the opening act goes on? No one knows.) And if you’re running late, you may want to find out if tonight’s big headliner has stepped onstage yet. The moment they do, tweets capturing the moment will flood the timeline. 

Twitter real-time search would know you’re asking about a currently scheduled concert, it should know where that concert is taking place based on posts from the artist and/or venue, and it can look through thousands of tweets from that location for the standard concert video format (blurry, dark, shaky). With this type of search, Twitter should be able to give you a clear, confident answer.

Mockup: When a particular query has larger amounts of clearer data to reference, Twitter’s real-time search AI could deliver more definitive results. 

“Why is there a crowd in front of this Foot Locker?”

I can’t tell you how many times I’ve wondered something like that. You’re walking down the street and notice a large group of people gathering for no discernible reason. Right now, there’s nowhere you can go to find out what’s happening. Twitter could (and should!) be that place.

“Is the bar across the street showing the Champions League final?”

Short-term business information, like events or holiday hours, is hard to find for consumers (and annoying for businesses to update). If a local bar posts about a special event on Twitter, real-time search can instantly find and verify it. Alternatively, if bar patrons watching the game post about it, real-time search can capture that.

Potential pitfalls

As excited as I am about the potential of real-time search on Twitter, I think it’s important to discuss the possible pitfalls of a tool like that. There are two significant issues I can think of, both of which relate to the reliability of the data used to provide real-time search results.

To discuss these challenges, let’s look at two similar events that both took place in 2016. Within a couple of weeks of each other, JFK and LAX both experienced false active shooter evacuations. Both situations escalated because of on-the-ground misunderstandings, but the panic spread rapidly because of Twitter. Real-time search could’ve made these events much worse. 

Pitfall #1: People aren’t always accurate sources of information

The faster the information, the more likely it is to be inaccurate. Even if people don’t mean to get the facts wrong, they still will. At JFK, a few people panicked because they mistook cheering for gunshots. Their panic spread to a few more people nearby. Naturally, they began tweeting. Those tweets reached other people across JFK, including the NYPD, TSA, and the FBI, prompting a wide-ranging security response, so everyone panicked. 

This is a tricky problem to solve. I could see AI playing a role in determining whether supposed events are true or false. Twitter should also contextualize search results to set accurate expectations. It would be fair to say that an emergency might be taking place at JFK, even if it didn’t involve a shooter. 

Pitfall #2: Evil robots and those who control them

What if this avalanche of false reports originated not from well-meaning, poorly informed people, but instead came from an army of bots designed to create chaos?

This is a much more concerning pitfall. Malicious parties could use real-time search on Twitter to spread false or misleading information. By using bots, they could flood Twitter with fake data and manipulate search results. If enough accounts tweet the same  information and a sufficient number of people at that location see it, real panic will ensue. That’s similar to what happened at LAX and JFK, but without the misunderstanding that started it all. 

The first step to prevent this malicious practice would be to base real-time search results on strict geolocation data, meaning that Twitter would only analyze and display tweets that were posted from the physical location being asked about in the original query. Twitter must also limit or blacklist bots from providing real-time search data. At this point, I’m not sure I trust their ability even to identify those bots. Either way, a lot of testing and a slow rollout would be needed to protect users. 

Twitter’s identity and future

Twitter’s identity crisis largely pre-dates Elon Musk’s ownership: It has always seemed like an app and company that struck gold and couldn’t quite figure out what to do with it. Experiments like Vine and Revue failed to provide a new direction for the service. I’ve assumed the team at Twitter has been asking themselves entirely the wrong questions, as they’ve searched for new features to extend beyond the tweet and timeline. Instead, they should look at what is inherently and uniquely valuable about them. 

Longtime users have always seemed to understand the power of Twitter, arguably more so than the company itself. Its identity, strategy, business model, even its survival, are up in the air right now. It’s hard to imagine that the answers lie in dolling out blue checkmarks for $8 a month. “Don’t miss what’s happening,” says the banner at the bottom of Twitter’s website, welcoming new users. If only it followed its own advice.

Author bio: Andre Plaut is the founder of human machine, a product strategy and learning design studio. He has spent over 10 years at the intersection of product, design, and education, developing and scaling learning experiences, products, and teams for organizations like Apple, Obama for America, General Assembly, Huge Inc, BARK, Indeed, and many others.