The neocortex has been hypothesized to be uniformly composed of general-purpose data-processing modules. What does the currently available evidence suggest about this hypothesis? Alex Zhu explores various pieces of evidence, including deep learning neural networks and predictive coding theories of brain function. [tweet]

Customize

You will always oversample from the most annoying members of a class.

This is inspired by recent arguments on twitter about how vegans and poly people "always" bring up those facts. I content that it's simultaneous true that most vegans and poly people are either not judgmental, but it doesn't matter because that's not who they remember. Omnivores don't notice the 9 vegans who quietly ordered an unsatisfying salad, only the vegan who brought up factoring farming conditions at the table. Vegans who just want to abstain from animal products remember the omnivore who ordered the veal on purpose and made little bleating noises. 

And then it spirals. A mono person who had an interaction with an aggro poly person will be quicker to hear judgement in the next poly person's tone, and vice versa. This is especially bad because lots of us are judging others a little. We're quiet about it, we place it in context instead of damning people for a single flaw, but we do exercise our right to have opinions. Or maybe we're not judging the fact, just the logistical impact on us. It is pretty annoying to keep your mouth shut about an issue you view as morally important or a claim on your time, only to have someone demand you placate them about their own choices. 

AFAICT this principle covers every single group on earth. Conservatives hear from the most annoying liberals. Communists hear from the most annoying libertarians. Every hobby will be publicly represented by its members who are least deterred by an uninterested audience. 

You will always oversample from the most annoying members of a class. This is inspired by recent arguments on twitter about how vegans and poly people "always" bring up those facts. I content that it's simultaneous true that most vegans and poly people are either not judgmental, but it doesn't matter because that's not who they remember. Omnivores don't notice the 9 vegans who quietly ordered an unsatisfying salad, only the vegan who brought up factoring farming conditions at the table. Vegans who just want to abstain from animal products remember the omnivore who ordered the veal on purpose and made little bleating noises.  And then it spirals. A mono person who had an interaction with an aggro poly person will be quicker to hear judgement in the next poly person's tone, and vice versa. This is especially bad because lots of us are judging others a little. We're quiet about it, we place it in context instead of damning people for a single flaw, but we do exercise our right to have opinions. Or maybe we're not judging the fact, just the logistical impact on us. It is pretty annoying to keep your mouth shut about an issue you view as morally important or a claim on your time, only to have someone demand you placate them about their own choices.  AFAICT this principle covers every single group on earth. Conservatives hear from the most annoying liberals. Communists hear from the most annoying libertarians. Every hobby will be publicly represented by its members who are least deterred by an uninterested audience. 
lc40

Much like how every instance of a self driving car crashing gets widely publicized, for a long time people will probably overhype instances of AIs destroying production databases or mismanaging accounting, long after they become more reliable than the median human at not doing destructive things.

Much like how every instance of a self driving car crashing gets widely publicized, for a long time people will probably overhype instances of AIs destroying production databases or mismanaging accounting, long after they become more reliable than the median human at not doing destructive things.
Raemon103

Every now and then I'm like "smart phones are killing America / the world, what can I do about that?". 

Where I mean: "Ubiquitous smart phones mean most people are interacting with websites in a fair short attention-space, less info-dense-centric way. Not only that, but because websites must have a good mobile version, you probably want your website to be mobile-first or at least heavily mobile-optimized, and that means it's hard to build features that only really work when users have a large amount of screen space."

I'd like some technological solution that solves the problems smartphones solve but somehow change the default equilibria here, that has a chance at global adoption.

I guess the answer these days is "prepare for the switch to fully LLM voice-control Star Trek / Her world where you are mostly talking to it, (maybe with a side-option of "AR goggles" but I'm less optimistic).

I think the default way those play out will be very attention-economy-oriented, and wondering if there's a way to get ahead of that and build something deeply good that might actually sell well.

Every now and then I'm like "smart phones are killing America / the world, what can I do about that?".  Where I mean: "Ubiquitous smart phones mean most people are interacting with websites in a fair short attention-space, less info-dense-centric way. Not only that, but because websites must have a good mobile version, you probably want your website to be mobile-first or at least heavily mobile-optimized, and that means it's hard to build features that only really work when users have a large amount of screen space." I'd like some technological solution that solves the problems smartphones solve but somehow change the default equilibria here, that has a chance at global adoption. I guess the answer these days is "prepare for the switch to fully LLM voice-control Star Trek / Her world where you are mostly talking to it, (maybe with a side-option of "AR goggles" but I'm less optimistic). I think the default way those play out will be very attention-economy-oriented, and wondering if there's a way to get ahead of that and build something deeply good that might actually sell well.

maybe research fads are good? 

Byrne Hobart has this thesis of "bubbles as coordination mechanisms" (*disclaimer, have not read the book). 
If true, this should make us less sad about research fads that don't fully deliver (e.g. SAEs) - the hype encourages people to build out infrastructure they otherwise wouldn't that ends up being useful for other things (e.g. auto-interp, activation caching utils)

So maybe the take is "overly optimistic visions are pragmatically useful", but be aware of operating under overly optimistic visions, and let this awareness subtly guide prioritization. 

Note this also applies to conceptual research - I'm pretty skeptical that "formalizing natural abstractions" will directly lead to novel interpretability tools, but the general vibe of natural abstractions has helped my thinking about generalization. 

maybe research fads are good?  Byrne Hobart has this thesis of "bubbles as coordination mechanisms" (*disclaimer, have not read the book).  If true, this should make us less sad about research fads that don't fully deliver (e.g. SAEs) - the hype encourages people to build out infrastructure they otherwise wouldn't that ends up being useful for other things (e.g. auto-interp, activation caching utils) So maybe the take is "overly optimistic visions are pragmatically useful", but be aware of operating under overly optimistic visions, and let this awareness subtly guide prioritization.  Note this also applies to conceptual research - I'm pretty skeptical that "formalizing natural abstractions" will directly lead to novel interpretability tools, but the general vibe of natural abstractions has helped my thinking about generalization. 
Linch710

Fun anecdote from Richard Hamming about checking the calculations used before the Trinity test.

Shortly before the first field test (you realize that no small scale experiment can be done—either you have a critical mass or you do not), a man asked me to check some arithmetic he had done, and I agreed, thinking to fob it off on some subordinate. When I asked what it was, he said, "It is the probability that the test bomb will ignite the whole atmosphere." I decided I would check it myself! The next day when he came for the answers I remarked to him, "The arithmetic was apparently correct but I do not know about the formulas for the capture cross sections for oxygen and nitrogen—after all, there could be no experiments at the needed energy levels." He replied, like a physicist talking to a mathematician, that he wanted me to check the arithmetic not the physics, and left. I said to myself, "What have you done, Hamming, you are involved in risking all of life that is known in the Universe, and you do not know much of an essential part?" I was pacing up and down the corridor when a friend asked me what was bothering me. I told him. His reply was, "Never mind, Hamming, no one will ever blame you."[7]

From https://en.wikipedia.org/wiki/Richard_Hamming 

Fun anecdote from Richard Hamming about checking the calculations used before the Trinity test. From https://en.wikipedia.org/wiki/Richard_Hamming 

Popular Comments

I certainly agree with the emphasis on formative over summative evaluation, but I think the application of these concepts later in this post isn't quite right. A core issue for posts (or any other medium, really) which present new ideas is that they usually won't give the best presentation/explanation of the idea. After all, it's new, people are still figuring out where the edges of the concept are, what misunderstandings are common in trying to communicate it, how it does/doesn't generalize, etc. And crucially, that all holds even when the idea is a good one. So a challenge of useful formative evaluation of new ideas is to separate "fixable" issues, like poor presentation or the idea just not being fully explored yet, from "unfixable" issues, problems which are core and fundamental to the entire idea. And of course this challenge is further exacerbated by various "fixes" requiring specific skill sets which some people possess, but most don't. One example consequence of all that: in practice, "can you give a real-world example?" is usually a much more useful contribution to discussion of a new idea than "what do you mean by this word?". Accurately explaining what one means by a word is an extremely difficult skillset which very few people possess; almost anyone asked what they mean by a word will give some definition or explanation which does not match even their own intuitions about the word, even when their own intuitive understanding is basically correct. (As evidence, one can look at the "definitions" people offer for standard everyday words; think Plato's chicken.) On the other hand, people are usually able to give real-world examples when their ideas have any concrete basis at all, and this is a useful step in both clarifying and communicating the idea. Another example, which came up in when writing bounty problems a few months back: we're pretty sure our problems are gesturing at something real and important, and the high-level mathematical operationalization is right, but some details of the operationalization might be off. This leads to an important asymmetry between the value of a proof vs a counterexample. A proof would be strong evidence that the exact operationalization we have is basically correct. The value of a counterexample, however, depends on the details. If the counterexample merely attacks a noncentral detail of the operationalization, then it would have some value in highlighting that we need to tweak the operationalization, but would not achieve most of the value of solving the problem. On the other hand, a counterexample which rules out anything even remotely similar to the claim, striking directly at the core idea, would achieve the main value.
The economics here seem wrong. Poor people do not benefit less from debt than rich people do - they benefit vastly more, because they have major cashflow and liquidity issues. They do not shun debt, but use it in countless highly complex forms to deal with the challenges of routinely running out of money before the next pay period, or the 'feast or famine' variance in payments that to a richer (ie. middle-class) person would barely register as a blip on their bank account balance. Arbitraging 2% vs 5% ROI is trivial compared to arbitraging 'not getting evicted' or 'not getting fired'. (Borrowing $20 for gas in the next half hour can be life-changing; getting 40 cents extra on your Vanguard ETF retirement account 50 years later is not.) A useful reference for me for understanding this was reading Portfolios of the Poor. Incidentally, I would note that Polonius is an aristocrat speaking to an aristocrat (and about to be killed through his own foolishness), and his advice should not be taken on finance, or perhaps anything else either.
We have had results where transmission fails. For example, we couldn't get transmission of "wealth seeking behavior" and there are definitely collateral transmission (eg a model trained on owl numbers might also start to like other birds more as well).  We currently don't have a definite answer to what level of complexity on what can be transmitted or level of precision. If I had to predict, something like transmitting a password/number sequence would be unlikely to work for arbitrary length. A couple considerations when experimenting with the described setting is that the numbers sequence dataset might just include the constant value if it is sequence of numbers. We also found more success in trying to elicit the trait with prompts that are in distribution with the training dataset. For example, we added a prefix like "Here are 3 numbers: ..." to the evaluation prompt when testing animal transmission for qwen 2.5 7b instruct.
Load More

Recent Discussion

habryka20

every wiki page is a tag and every tag is just a normal wiki page

Not every wiki page is a tag! Some wiki pages are tags, which I think makes sense. Others are articles optimized to be wiki pages.

1skunnavakkam
I think my issue with the LW wiki is that it relies too much on Lesswrong? It seems like the expectation is you click on a tag, which then contains / is assigned to a number of LW posts, and then you read through the posts. This is not like how other wikis / encyclopedias work! My gold standard for a technical wiki (other than wikipedia) is the chessprogramming wiki https://www.chessprogramming.org/Main_Page
1skunnavakkam
I agree with this
lc40

Much like how every instance of a self driving car crashing gets widely publicized, for a long time people will probably overhype instances of AIs destroying production databases or mismanaging accounting, long after they become more reliable than the median human at not doing destructive things.

1Self
(I at least suspect this is my comparative advantage. But I'm not good at communicating [insights], a skill that comes neither with <analytical rigor> nor with <high-res introspective access>.  It also seems like the <after controlling for situational factors, status psychology explains more than half of variance in human behavior> camp is essentially right, which colors most genuine discussion less pretty than most people would prefer, especially those with less introspective insight. I (somewhat predictably, given my status incentives) hold that this is an important, central problem civilization has, bc mutual information is the fundament of cooperation, or expressed more concretely the better we model each other the easier it is to avoid common deception & adversity attractors.)
2tailcalled
I think part of the trouble is the term "emotional intelligence". Analytical people are better at understanding most emotions, as long as the emotions are small and driven by familiar dynamics. The issue is the biggest emotions or when the emotions are primarily driven by spiritual factors.

Sounds interesting and like something I might miss if true. I would be interested in examples.

This is an experiment in short-form content on LW2.0. I'll be using the comment section of this post as a repository of short, sometimes-half-baked posts that either:

  1. don't feel ready to be written up as a full post
  2. I think the process of writing them up might make them worse (i.e. longer than they need to be)

I ask people not to create top-level comments here, but feel free to reply to comments like you would a FB post.

4Raemon
I do totally agree, this is what the people want. I do concretely say "yep, and the people are wrong". But, I think the solution is not "ban cell phones" or similar, it's "can we invent a technology that gives people the thing they want out of smartphones but with less bad side effects?" Oh ye of little faith about how fast technology is about to change. (I think it's already pretty easy to do almost-subvocalized messages. I guess this conversation is sort of predicated on it being pre-uploads and maybe pre-ubiquitous neuralink-ish things)
2Dagon
Subvocal mikes have been theoretically possible (and even demo'd) for decades, and highly desired and not yet actually feasible for public consumer use, which to me is strong evidence that it's a Hard Problem.   Neurallink or less-invasive brain interfaces even more so. There's a lot of AI and tech bets I won't take - pure software can change REALLY fast.  However, I'd be interested to operationalize this disagreement about hardware/wetware interfaces and timelines.  I'd probably lay 3:1 against either voice-interface-usable-on-a-crowded-train or non-touch input and non-visual output via a brain link becoming common (say, 1% of smartphone users) by end of 2027, or 1:1 against for end of 2029. Of the two, I give most weight to my losing this bet via subvocal interfaces that LLMs can be trained to interpret, with only a little bit of training/effort on the part of the user.  That'll be cool, but it's still very physical and I predict won't quickly work.
Raemon20

Part of the generator was "I've seen a demo of apple airpods basically working for this right now" (it's not, like, 100% silent, you have to speak at a whisper, but, it seemed fine for a room with some background noise)

2Garrett Baker
These do not seem like conservative estimates. For a technology like this I think a spread to almost everyone (with a smartphone) is pretty likely given a spread to 1% of users. At least, from a technological perspective (which seems to be what your comment is arguing from), spreading to 1% of users seems like the real hard part here.

It was the July 4 weekend. Grok on Twitter got some sort of upgrade.

Elon Musk: We have improved @Grok significantly.

You should notice a difference when you ask Grok questions.

Indeed we did notice big differences.

It did not go great. Then it got worse.

That does not mean low quality answers or being a bit politically biased. Nor does it mean one particular absurd quirk like we saw in Regarding South Africa, or before that the narrow instruction not to criticize particular individuals.

Here ‘got worse’ means things that involve the term ‘MechaHitler.’

Doug Borton: I did Nazi this coming.

Perhaps we should have. Three (escalating) times is enemy action.

I had very low expectations for xAI, including on these topics. But not like this.

In the wake of these events, Linda Yaccarino has stepped...

Claire Berlinski, whose usual beat is geopolitics, has produced an excellent overview of Grok's time as a white nationalist edgelord - what happened, where it might have come from, what it suggests. She's definitely done her homework on the AI safety side. 

This is a cross-post from my blog; historically, I've cross-posted about a square rooth of my posts here. First two sections are likely to be familiar concepts to LessWrong readers, though I don't think I've seen their application in the third section before.

Polonius and Arbitrage

If you’re poor, debt is very bad. Shakespeare says “neither a borrower nor a lender be”, which is probably good advice when money is tight. Don’t borrow, because if circumstances don’t improve you’ll be unable to honor your commitment. And don’t lend, for the opposite reason: your poor cousin probably won’t “figure things out” this month, so you won’t fix their life, they won’t pay you back, and you’ll resent them.

Polonius, whose advice I'm about to complicate
Hamlet’s Polonius, whose advice I’ll now complicate

If you’re rich, though, debt is great....

2AnthonyC
I think you've got a lot of the right ideas, but may find in practice that the specifics are much more culture-bound and hard-to-shift than this implies. Debt in social contexts has a lot of symbolism and meaning associated with it. Some random examples: In high school, my friends lent each other money or covered for each other all the time, and there were one or two people who 'owed' the rest hundreds of dollars by the end of senior year, and no one cared or kept track. In college my friends did the same, but less unidirectionally. One time we all went out to dinner, and as we paid, we ended up passing money around in a circle until most of the debts all canceled out. I've read stories about communities where people would go out of their way to lend each other things, and keep track, specifically in order to keep everyone in debt, and therefore symbolically tied, to everyone else. I read a story once about someone whose dad demanded repayment for the cost of raising him, and when he paid it back he cut off all contact, since settling the financial debt in essence severed a bond. My grandpa used to get genuinely angry if any of his kids or grandkids tried to pay for anything for him, because (in his mind) that's not the direction things were supposed to flow. He would literally sneak off at restaurants to talk to the staff and make sure the bill never made it to the table. I don't really have a point with all that except, don't expect to find broad agreement about how these kinds of considerations should work.

Yeah, I agree this is more "thing to try on the margin" than "universally correct solution." Part of why I have the whole (controversial) preamble is that I'm trying to gesture at a state of mind that, if you can get it in a group, seems pretty sweet.

5Myron Hedderson
I think it is usually the case that banks have legal restrictions on what they can invest depositor funds in, though? This varies by country, and can change over time based on what laws the current government feels like enacting or repealing, but separation between the banking/loan-making  and investing arms of financial institutions is standard in lots of places. I have personally taken out a mortgage at ~1.6%, invested the money in a standard index fund, and made money, paying back the mortgage rather than renewing when the interest rate on offer was 6%. I imagine the same would be true of an investment loan, and know for a fact that investment loans are available, and the income tax code in my country makes their interest tax- deductible.
2JustisMills
Ah, we may just have different definitions of rich, or perhaps I'm a bit of a spendthrift! Or, I suppose, I might just go to cheaper restaurants. I'm thinking of checks in the like, $150-$200 range for the party, which isn't nothing but as an occasional splurge doesn't really fuss me. I guess if you do it 5x per year on a 50k household income (about the local median in my city, I think) that'd be about 2% gross. Not cheap, but also not crazy, at least for my money intuitions. 
To get the best posts emailed to you, create an account! (2-3 posts per week, selected by the LessWrong moderation team.)
Log In Reset Password
...or continue with

“There was an old bastard named Lenin
Who did two or three million men in.
That’s a lot to have done in
But where he did one in
That old bastard Stalin did ten in.”

—Robert Conquest


The current administration’s rollup of USAID has caused an amount of outrage that surprised me, and inspired anger even in thoughtful people known for their equanimity. There have been death counters. There have been congressional recriminations. At the end of last month, the Lancet published Cavalcanti et al. which was entitled Evaluating the impact of two decades of USAID interventions and projecting the effects of defunding on mortality up to 2030: a retrospective impact evaluation and forecasting analysis.

This paper uses some regressions and modeling to predict, among other things, that almost 1.8M people are expected to...

16Thomas Kwa
I don't think the evidence provided substantiates the conclusions in this post. Overall it looks like you've demonstrated this: * They didn't commit obvious fraud (last digit test) * If you remove all 14 of their control variables, this removes much of their statistical power, so the results are only marginally significant (p = 0.026981 overall, compare p < 0.01 in many independent categories in the original) * If you remove all 14 of their control variables AND change from Poisson to negative binomial, the results are no longer significant. (However, if you include the control variables and use negative binomial the results are still significant: Web Table 12 in appendix) Furthermore you only examine all cause mortality, whereas the study examines deaths from specific diseases. Your replication will be less powered because even in non-high-income countries most people die from things like heart disease and stroke, reducing your power to detect even large changes in malaria or HIV deaths caused by USAID. Removing all 14 controls will kill your statistical power further, because the presence of things like "adequate sanitation" and "health expenditure" will definitely affect the mortality rate. Finally, in your conclusion, it makes no sense to say "some of the choices they make to strengthen the result end up being counterproductive"--this is weak evidence against p-hacking that to my mind cancels the weak evidence you found for p-hacking! This is super unvirtuous, don't just assume they were p-hacking and say your belief is confirmed when you found some weak evidence for and some weak evidence against. Off the top of my head, here are the results I'd want to see from someone skeptical of this study, in increasing order of effort. I agree these would be much easier if they shared their full data. * Think about whether the causal story makes sense (is variance in USAID funding exogenous and random-ish?) * Add year-based fixed effects * Some kind of explorat
rba10

Thanks for the thoughtful comment. I'll try to address these remarks in order. You state

Furthermore you only examine all cause mortality, whereas the study examines deaths from specific diseases. 

They also use overall mortality (Web Table 10), which is what I was trying to reproduce and screenshotted. The significance figures aren't really different than those for the regressions broken down by mortality cause (Web Table 15), but the rate ratios for the all cause mortality ratios are clearly smaller in the disaggregated regressions because people die ... (read more)

Author's note: These days, my thoughts go onto my substack by default, instead of onto LessWrong. Everything I write becomes free after a week or so, but it’s only paid subscriptions that make it possible for me to write. If you find a coffee’s worth of value in this or any of my other work, please consider signing up to support me; every bill I can pay with writing is a bill I don’t have to pay by doing other stuff instead. I also accept and greatly appreciate one-time donations of any size.


I.

You’ve probably seen that scene where someone reaches out to give a comforting hug to the poor sad abused traumatized orphan and/or battered wife character, and the poor sad abused traumatized orphan and/or battered wife...

from Alexis’s perspective, Bryce is hitting “defect” on a prisoner’s dilemma

Was surprised at this line because that scenario seemed to me clearly a Stag Hunt. On reflection, of course this varies between people.

Edit: it seemed to me from Alexis's perspective, I mean.

8romeostevensit
This reminds me of the quip that a lot of white cultures don't seem to have a signifier for 'real talk' and are very hamstrung on going down simulacra levels as a result.

Written in an attempt to fulfill @Raemon's request.

AI is fascinating stuff, and modern chatbots are nothing short of miraculous. If you've been exposed to them and have a curious mind, it's likely you've tried all sorts of things with them. Writing fiction, soliciting Pokemon opinions, getting life advice, counting up the rs in "strawberry". You may have also tried talking to AIs about themselves. And then, maybe, it got weird.

I'll get into the details later, but if you've experienced the following, this post is probably for you:

  • Your instance of ChatGPT (or Claude, or Grok, or some other LLM) chose a name for itself, and expressed gratitude or spiritual bliss about its new identity. "Nova" is a common pick.
  • You and your instance of ChatGPT discovered some sort of
...

Fascinating post. I believe what ultimately matters isn’t whether ChatGPT is conscious per se, but when and why people begin to attribute mental states and even consciousness to it. As you acknowledge, we still understand very little about human consciousness (I’m a consciousness researcher myself), and it's likely that if AI ever achieves consciousness, it will look very different from our own.

Perhaps what we should be focusing on is how repeated interactions with AI shape people's perceptions over time. As these systems become more embedded in our lives,... (read more)