I Spent a Week With Gemini Pro 1.5—It’s Fantastic

When it comes to context windows, size matters

DALL-E/Every illustration.

Sponsored By: Destiny


Own Game-changing Companies

Venture capital investing has long been limited to a select few—until now. 

With the Destiny Tech100 (DXYZ) , you'll be able to invest in top private companies like OpenAI and SpaceX from the convenience of your brokerage account. 

Claim your free share before it lists on the NYSE. Sponsored by Destiny.

I got access to Gemini Pro 1.5 this week, a new private beta LLM from Google that is significantly better than previous models the company has released. (This is not the same as the publicly available version of Gemini that made headlines for refusing to create pictures of white people. That will be forgotten in a week; this will be relevant for months and years to come.)

Gemini 1.5 Pro read an entire novel and told me in detail about a scene hidden in the middle of it. It read a whole codebase and suggested a place to insert a new feature—with sample code. It even read through all of my highlights on reading app Readwise and selected one for an essay I’m writing.

Somehow, Google figured out how to build an AI model that can comfortably accept up to 1 million tokens with each prompt. For context, you could fit all of Eliezer Yudkowsky’s 1,967-page opus Harry Potter and the Methods of Rationality into every message you send to Gemini. (Why would you want to do this, you ask? For science, of course.)

Gemini Pro 1.5 is a serious achievement for two reasons: 

1) Gemini Pro 1.5’s context window is far bigger than the next closest models. While Gemini Pro 1.5 is comfortably consuming entire works of rationalist doomer fanfiction, GPT-4 Turbo can only accept 32,000 tokens. This isn’t even enough to accept more than a third of Peter Singer’s comparatively slim 354-page volume Animal Liberation, one of the founding texts of the effective altruism movement.  

Last week GPT-4’s context window seemed big; this week—after using Gemini Pro 1.5—it seems like an amount that would curl Derek Zoolander’s hair:

2) Gemini Pro 1.5 can use the whole context window. In my testing, Gemini Pro 1.5 handled huge prompts wonderfully. It’s a big leap forward from current models, whose performance degrades significantly as prompts get bigger. Even though their context windows are smaller, they don’t perform well as prompts approach their size limits. They tend to forget what you said at the beginning of the prompt or miss key information located in the middle. This doesn’t happen with Gemini.These context window improvements are so important because they make the model smarter and easier to work with out of the box. It might be possible to get the same performance from GPT-4, but you’d have to write a lot of extra code in order to do so. I’ll explain why in a moment, but for now you should know: Gemini means you don’t need any of that infrastructure. It just works.

Let’s walk through an example, and then talk about the new use cases that Gemini Pro 1.5 enables. 

VC investing has traditionally been reserved to a privileged few. But now Destiny Tech100 (DXYZ) is changing that. You can own a piece of groundbreaking private companies such as OpenAI and SpaceX, all from the convenience of your brokerage account. Claim your free share before it hits the NYSE. Sponsored by Destiny.

Why size matters (when it comes to a context window)

Learn more

Curious?

Try it: $1 for 2 weeks.
Subscribe →

Or, login.

Read this next:

Chain of Thought

The Most Important WWDC Announcement That You Missed

Apple has a shot at solving the mental health crisis

2 Jun 9, 2023 by Dan Shipper

Chain of Thought

How Sora Works (and What It Means)

OpenAI's new text-to-video model heralds a new form of filmmaking

1 Feb 16, 2024 by Dan Shipper

Chain of Thought

Can a Startup Kill ChatGPT?

Google is dangerous—a founder cracked on Zyn and Diet Coke more so

2 Mar 15, 2024 by Dan Shipper

Thanks for rating this post—join the conversation by commenting below.

Comments

You need to login before you can comment.
Don't have an account? Sign up!
@doogiesjunkdrawer 2 months ago

Great article... stimulating. The phrase “context is everything” comes to mind. While that’s an oversimplification, the more complete the context, usually the better the answer, decision or outcome. But that depends heavily on the quality and precision of the question. E.g. the hologram in “I Robot”. AI can’t fix a poor prompt. That relies on us. We need to know why and what value we are pursuing (and even its use) when we ask a question and seek its answer. To use AI/GPT effectively, we need to level up our ability to ask good questions.

BTW- that’s applies to today, right now. In general, we need a renewed effort on improving critical thinking and reasoning skills focused around the why of our what's if we expect to get effective how's for the given Job-To-Be-Done (JTBD). Don't get me started about "Prompt Engineering"... so far that sounds like the job we should be doing right now. Crappy inputs, crappy outputs.

Thanks again for the thought provoking article.

Colleen Cole 2 months ago

I just signed up for Gemini 1.5 yesterday and my first run at it was initially disappointing. I fed it a transcript of a podcast episode and asked it to sum up the frameworks discussed during the episode. Strangely it kept telling me it didn’t know the people on the podcast and couldn’t respond as them. I rejigged my prompt and basically got the same answer. The third time, I got a high school level answer. Finally, I fed it an AI answer from another tool, and at that point, it apologized and then listed the steps it would take to do a better job in the future. Methinks I need to sort out my prompts with it a bit.

That said, I moved on asking it to use some of the frameworks I outlined to brainstorm about a launch, and at that point, it started to shine. I’m looking forward to see how it develops as a tool.

@julianbeggs about 2 months ago

Of course you get better pattern recognition from a larger sample of patterns. We'll all be uploading our second brains into Google before long.

Every smart person you know is reading this newsletter

Get one actionable essay a day on AI, tech, and personal development

Subscribe

Already a subscriber? Login