People prefer giving orders to taking them. They prefer granting permission (or not) to having to ask permission. They prefer questioning others over being questioned. They prefer more pressure on others to answer to them, and less pressure on themselves to answer to others.

People try to figure out how to achieve these things. This begins in early childhood when they discover that their parents have social (and physical) power over them. Then they have to answer to their teachers, to babysitters, to the parents at a friend’s house they’re visiting, to adult relatives, and more.

They find at the peer level that some people get what they want more, are listened to more, are respected more, and so on. Doing better with peers is realistically achievable in the short term.

They find they look weak when they’re insecure, needy, unconfident, reactive, seeking approval from others, acting like a follower not a leader, etc. They learn how to act to get others to put in more effort than they do, to have people come to them, to get approval while looking like you aren’t trying to get approval, to hide their effort, to make comments highlighting others’ weaknesses without being considered an aggressor, to recognize and follow trends a little on the early side. They learn to hide weakness and ignorance. They learn to be dishonest. Some things are not even really considered dishonest, socially, they’re just normal. But they aren’t how a scientist thinks. A scientist volunteers relevant info in pursuit of truth instead of looking for opportunities to withhold unfavorable info. And in short, doing anything less than an idealized scientist would is dishonest to some extent.

People discover there are both formal and informal social power structures. Being a teacher, parent or boss is a formal position. It’s an explicit label. The leader of a group of friends is an informal position. There’s no contract or clear rules. It can just change as opinions change.

Sometimes formal and informal social power have a mismatch. A general may not have the respect of the soldiers he gives orders to. A boss may struggle to get his subordinates to listen to him – formally he’s in charge but informally people don’t see him that way.

Mismatches aren’t terribly common. People whose informal social power is significantly below their formal position often get replaced. More often, people don’t get positions in the first place if they don’t have an appropriate informal social status. Mismatches are often caused by giving out positions due to favoritism instead of merit, e.g. getting a position for one’s child who didn’t (socially) earn it (and often didn’t earn it on objective world merit, like knowledge and skill, either). Sometimes mismatches develop over time – things started out OK but a person lost respect or got undermined or something over time. Status can be unstable. People often get promotions to positions they’re expected to probably be able to handle, but there’s no guarantee and it doesn’t always work out.

However, mismatches are extremely common when no one cares what the subordinates think or want. If the subordinates are there voluntarily – e.g. customers or people who could get a different job or transfer to a different division in the company – it puts pressure against mismatches. However, when the subordinates are children, prison inmates, involuntary psychiatry patients, or the elderly in an old folks home, then the people in power may be hated by their subordinates and stay in power anyway. This is also a big problem with the government and its citizens – there is some accountability but generally not enough.

Anyway, people learn how to behave so they do well in terms of informal social status. There are incentives and benefits there, and it’s also one of the main things that leads to gaining and keeping formal social status.

Communications and actions have two main interpretations: social and objective.

Objective interpretations look at the literal meaning. They use rational and scientific analysis. They try to avoid logical errors. They aim to account for all the evidence and contradict none. They don’t judge the truth of ideas by the attributes of the person who thought of or communicated the idea.

Social interpretations consider the speaker or actor in relation to other people. What is he trying to do to or get from others? How does the action/communication affect the status of the actor and others? Is someone being needy or reactive? Is someone showing weakness? Is someone socially attacking someone else? Is someone becoming more or less connected with something high or low status (e.g. tribe allegiance signals).

Social interpretations are allowed to focus on some evidence and ignore or contradict other evidence. They can be illogical and unscientific. Some evidence is impolite to use or mention. Some conclusions are jumped to on a basis like “since the social meaning of that action/communication is so strong and obvious, you must have done it intentionally and chosen that social meaning on purpose, no matter what you say about a misunderstanding or that you were focusing on an objective goal.”

Example (try to read hyper literally, and look at this really logically, in order to understand Sue’s perspective):

Joe: I don’t understand what you said.
Sue: What are you planning to do about that?
Joe: I just asked you to explain.
Sue: You didn’t make a request or ask a question.
Joe: I just did. wtf!

Joe is focused on the social world while Sue is in objective, logical mode. Joe expects that, when he speaks, Sue will guess what he wants from her and why he said it. Joe takes this for granted so much that he doesn’t notice the difference between him asking for something explicitly or via hints – he uses hints and thinks he asked. Sue thinks Joe’s statements provide information and that it isn’t her job to read Joe’s mind. Mind reading always runs into a bunch of ambiguity that’s hard to make guesses about, and the whole point of a conversation is for people to communicate their ideas themselves.

The conversation can easily get worse. Let’s continue it:

Sue: Quote?
Joe: “I don’t understand what you said.”

This is actually somewhat unrealistic. People like Joe often won’t quote at all or will misquote. Often they ambiguously explain which message like “the statement we’re talking about” or “the message that starts with ‘I’”. Sometimes they say that asking for quotes is unreasonable or unnecessary, or they stop replying. But let’s not get distracted by those problems.

Sue: Is that an imperative or interrogative sentence?
Joe: No.
Sue: What are the verb and grammatical subject?
Joe: “do understand” and “I”.

These are atypically accurate and patient grammar answers by Joe. Things could have gotten a lot worse here. They’re atypical because they’re objective-mode answers, not social-mode answers.

The social-mode meaning of “Is that an imperative or interrogative sentence?” is that Sue is acting like a teacher and putting Joe in an inferior student role. Sue asks the questions and Joe has the role of being questioned. Sue can initiate things of her choice and Joe has to react to Sue’s whims. She’s pushing this kind of framing on the situation. So typically someone like Joe will avoid answering the question in order to deny and push back against that framing. He’ll try to get Sue answering his questions or otherwise establish social power over her. He’ll avoid compliance on purpose. He might say something especially sophisticated to try to prove how grown up he is. He’ll think Sue is calling him dumb by asking him a fairly basic grammar question about the sort of thing he was supposed to have learned in school over 10 years ago.

Sue: So isn’t it a declarative statement about yourself?
Joe: You knew what I meant.

This doesn’t answer Sue’s question. It’s also asking for mind reading. It’s the sort of thing that only works with similar people. It’s a reasonable guess about most people from Joe’s subculture (they knew what he wanted and are being difficult on purpose), though there’s evidence throughout that Sue has a pretty different perspective on the world than Joe and genuinely found it problematic to assume instead of relying on communication. Joe’s attitude makes conversation very hard with people very different from himself. It’s bad at engaging with other frameworks or points of view.

Asking multiple questions in a row amplifies the teacher/student dynamic. That increases the pressure on Joe to break out of it. Sue isn’t thinking about the social meaning of what she says, so she doesn’t control it, but Joe keeps looking for it and reacting to it. If Sue were to consider the social meaning of each of her statements before saying it, she’d find it much harder to converse. Like if asking clarifying questions is socially aggressive (both in a “you answer to me” sense and a “you were unclear” accusation sense), what should she do to fix it? You can’t just skip clarifying questions in general. And how many are needed is out of her control. Sue can try to minimize the number of clarifying questions Joe needs to ask her, but it’s up to Joe to minimize how many Sue needs to ask him.

Sue: I thought I did. I thought you were providing information about the state of your understanding.
Joe: I was asking for help.
Sue: But that isn’t what your words mean. Why don’t you use standard English to say what you mean?

Joe feels highly insulted. But from an objective perspective, it’s a reasonable thing to be wondering and talking about. Joe literally says X and then acts like he’d communicated Y. Why not just say what he means?

Joe: Why don’t you just put two and two together?

Joe doesn’t ask the question, asks a counter-question that’s socially insulting to Sue (it implies she’s being dumber than like a 4 year old who can’t add 2+2 correctly)

Sue: There are dozens of reasonable ways to proceed given the information you provided. That’s why I asked which one you were planning.

Even with such an insulting question that wasn’t meant to be answered, Sue still takes it at face value and tries to explain the answer.

Joe: Why won’t you just tell me what you meant?
Sue: You haven’t asked me to.

Again Sue immediately answers Joe’s question. She’s responsive and still operating with good will. She doesn’t care about who is reacting to who and how it looks for social power. And Joe’s tilted (since Sue’s first or second message in the example dialog) but Sue isn’t.

Joe: I just did.
Sue: When?
Joe: Right now. I just asked.
Sue: Quote?
Joe: “Why won’t you just tell me what you meant?”
Sue: I answered that question.

This is similar to how the dialog started. Things haven’t been sorted out. Even though Sue explained her perspective earlier, Joe still isn’t taking it into account and adjusting his communications and expectations. Sue, meanwhile, doesn’t know what to change to make things work better. She knows she’s logically right. She thinks Joe ought to try discussing in a way that isn’t logically wrong and that it’s easier to have conversations which build on that foundation. If Joe doesn’t have the skill to do that, he should try to learn it and ask for help instead of trying to have a conversation he’s incapable of handling productively.

Joe: After all this, I can’t get any answers out of you. Goodbye forever.

Objectively, Sue did answer all of Joe’s questions and was responsive to all direct, explicit requests. But Sue kept ignoring the social world meanings of both Joe’s and her own words.

In the social world, direct requests are often too pushy. People often phrase statements as questions to weaken them (“That is a dog?” which is expressing some uncertainty and making it easier for the person to disagree or confirm) and questions as statements (“I wonder if that’s a dog.” which is asking if the other person thinks it’s a dog).

Sue: I don’t understand why people come to discussion forums when they clearly don’t have a basic grasp of English and logic, and they also aren’t aiming to learn those things. What they’re doing will never work.
Joe: wtf! Leave me alone.
Sue: I did. I didn’t expect you to return. I’m just post morteming my discussion and hoping someone else may have insight. This has nothing to do with you.
Joe: You’re flaming me and attacking my reputation.
Sue: I’m just analyzing public evidence. If I made an error you can point it out. I’m not trying to flame; I’m aiming for accuracy. If you don’t want people to think about what you say, don’t post it. If you want to look good when analyzed, improve your skill level. Now leave me in peace. I’ve got several more thoughts to post.
Joe: You have no right! I didn’t sign up to be treated this way!
Sue: People thinking about and discussing things you said is exactly what you signed up for when you posted them.
Joe: [Leaves and holds a long-lasting grudge.]
Sue: When he said “You knew what I meant.” I was tolerant, lenient and generous by letting him change topics in the middle of my grammar point. Right as I was getting to a conclusion he ignored my question, seemingly because he knew he was about to lose the debate. I wasn’t rewarded for being so helpful. He didn’t reciprocate with good will towards me. Maybe I would have been better off repeating my question until he answered it, or pointing out that he wasn’t answering.
Sue: I don’t understand how, after a long conversation about how he hadn’t made a particular request, he still didn’t get it enough to realize he still hadn’t actually made that request. Which new statement did he think constituted a request to explain something to him?
Sue: Does anyone understand why most people are like this? I have such good will but it never seems to be enough. On my initiative, I asked about his plans, thinking perhaps to help with them. My reward was that he derailed the conversation. And as usual he doesn’t want to discuss what went wrong. He did begin the process of clearing up misunderstandings, but he was creating new misunderstandings faster than we could resolve stuff. Why won’t people just calm down and use English in a simple, correct way? Why do they rush through conversations and make huge messes and then give up?
Sue: What is life like for such a person? Do none of his conversations work? What happens when two Joes talk? Is it pure chaos or does it seem to work, somehow? Maybe if they both say and want sufficiently stereotyped things they can stay on the same page with almost no communication, merely by assuming stereotypes.
Sue: Why is no one else talking? Is this a dead forum? Does no one here care about trying to understand how to have rational conversations with typical people? Don’t you guys run into problems like these and want to figure out what to do about them? Or are you all similar to Joe and hiding it with your silence?
Joe: [Reads all this and intensifies his grudge.]

It could easily go worse than this more quickly. But I wanted to draw it out a bit and show the ongoing perspective clashes.

The other people on the forum are more attuned to the social world than Sue.

Sue doesn’t think that attuning to the social world more herself will actually result in intellectual progress on topics like science, epistemology, AGI, etc. People need to think objectively to contribute to those topics anyway.

Routinely, people’s social status is inaccurate in some way. Then the truth threatens the status. What could Sue do when talking to people who need to admit weakness and try to learn some stuff but who prefer to dishonestly pretend to be better than they are and who don’t want Sue to speak the truth? What’s to be done with such people besides detecting and ignoring them (and maybe criticizing them if they’re public figures)?

Lots of people try to debate sorta like Sue but they aren’t actually very good at logic themselves. Joe has experience with that. He’s dealt with people who are mostly focused on social, and screw up logic, but they do some Sue stuff as an act. He assumes Sue is like that too. He doesn’t actually have the skill to judge whether Sue got anything wrong or not. But Joe thinks he does. So what happens is Joe will misunderstand something, think Sue made a mistake, and conclude that Sue isn’t as logical as she thinks she is. From Joe’s pov, Sue seems to be about as good at logic as Joe or worse – because whenever she uses superior skill there’s a good chance that Joe doesn’t get it, and when Joe doesn’t get it there’s a good chance that he attributes the error to Sue. (These things are not a matter of random chance. But if you look at many similar events, you’ll find sometimes it goes one way and sometimes the other. And it’s too hard to analyze the detailed causality.)

Most real discussions have a lot of intentional social by both parties. This is a stylized example that turns up the contrast between characters. Actually what sort of social counts as “intentional” is a tricky question. Most of it is automated in childhood. Most social by adults is done without conscious intention at the time they do it. So “I wasn’t trying to social you” is no defense – in the past you learned how to do that kind of social to people and practiced it until it was second nature. You’re responsible for that! This explains part of why people have trouble turning it off. And it explains why people assume that most social is intentional in the sense of there was an intent in the past when the person learned to do it. They aren’t doing it now randomly or accidentally. The main excuse is “autism” which is the most standard term for a person who didn’t learn, practice and automate a bunch of social dynamics (or somehow managed to stop and change), so then they might actually honestly not be playing the social game.

This is all complicated by the social rules of evidence. Lots of social dynamics are deniable even when everyone knows they were done on purpose. You did them right and people approve, so you’re allowed to get away with them rather than be called out for the social manipulations (if the call out is sufficiently socially savvy it can often work, but just a blunt, direct logic-focused callout doesn’t work). So it’s common that everyone knows social happened and it wasn’t an accident, but everyone pretends not to know.

stucchio and Mason

stucchio retweeted Mason writing:

"Everything can be free if we fire the people who stop you from stealing stuff" is apparently considered an NPR-worthy political innovation now, rather than the kind of brain fart an undergrad might mumble as they come to from major dental work https://twitter.com/_natalieescobar/status/1299018604327907328

There’s no substantial objective-world content here. Basically “I disagree with whatever is the actual thing behind my straw man characterization”. There’s no topical argument. It’s ~all social posturing. It’s making assertions about who is dumb and who should be associated with what group (and, by implication, with the social status of that group). NPR-worthy, brain fart, undergrad, mumble and being groggy from strong drugs are all social-meaning-charged things to bring up. The overall point is to attack the social status of NPR by associating it with low status stuff. Generally smart people like stuchhio (who remains on the small list of people whose tweets I read – I actually have a pretty high opinion of of him) approve of that tribalist social-political messaging enough to retweet it.

Yudkowsky

Eliezer Yudkowsky wrote on Less Wrong (no link because, contrary to what he says, someone did make the page inaccessible. I have documentation though.):

Post removed from main and discussion on grounds that I've never seen anything voted down that far before. Page will still be accessible to those who know the address.

The context is my 2011 LW post “The Conjunction Fallacy Does Not Exist”.

In RAZ, Yudkowsky repeatedly brings up subculture affiliations he has. He read lots of sci fi. He read 1984. He read Feynman. He also refers to “traditional rationality” which Feynman is a leader of. (Yudkowsky presents several of his ideas as improvements on traditional rationality. I think some of them are good points.) Feynman gets particular emphasis. I think he got some of his fans via this sort of subculture membership signaling and by referencing stuff they like.

I bring this up because Feynman has a book title "What Do You Care What Other People Think?": Further Adventures of a Curious Character. This is the sequel to the better known "Surely You're Joking, Mr. Feynman!": Adventures of a Curious Character.

Yudkowsky evidently does care what people think and has provided no indication that he’s aware that he’s contradicting one of his heroes, Feynman. He certainly doesn’t provide counter arguments to Feynman.

Downvotes are communications about what people think. Downvotes indicate dislike. They are not arguments. They aren’t reasons it’s bad. They’re just opinions. They’re like conclusions or assertions. Yudkowsky openly presents himself as taking action because of what people think. It’s also basically just openly saying “I use power to suppress unpopular ideas”. Yudkowsky also gave no argument himself, nor did he endorse/cite/link any argument he agreed with about the topic.

Yudkowsky is actually reasonably insightful about social hierarchies elsewhere, btw. But this quote shows that, in some major way, he doesn’t understand rationality and social dynamics.

Replies to my “Chains, Bottlenecks and Optimization”

https://www.lesswrong.com/posts/Ze6PqJK2jnwnhcpnb/chains-bottlenecks-and-optimization

Dagon

I think I've given away over 20 copies of _The Goal_ by Goldratt, and recommended it to coworkers hundreds of times.

Objective meaning: I took the specified actions.

Social meaning: I like Goldratt. I’m aligned with him and his tribe. I have known about him for a long time and might merit early adopter credit. Your post didn’t teach me anything. Also, I’m a leader who takes initiative to influence my more sheep-like coworkers. I’m also rich enough to give away 20+ books.

Thanks for the chance to recommend it again - it's much more approachable than _Theory of Constraints_, and is more entertaining, while still conveying enough about his worldview to let you decide if you want the further precision and examples in his other books.

Objective meaning: I recommend The Goal.

Social meaning: I’m an expert judge of which Goldratt books to recommend to people, in what order, for what reasons. Although I’m so clever that I find The Goal a bit shallow, I think it’s good for other people who need to be kept entertained and it has enough serious content for them to get an introduction from. Then they can consider if they are up to the challenge of becoming wise like me, via further study, or not.

This is actually ridiculous. The Goal is the best known Goldratt book, it’s his best seller, it’s meant to be read first, and this is well known. Dagon is pretending to be providing expert judgment, but isn’t providing insight. And The Goal has tons of depth and content, and Dagon is slandering the book by condescending to it in this way. By bringing up Theory of Constraints, Dagon is signaling he reads and values less popular, less entertaining, less approachable non-novel Goldratt books.

It's important to recognize the limits of the chain metaphor - there is variance/uncertainty in the strength of a link (or capacity of a production step), and variance/uncertainty in alternate support for ideas (or alternate production paths).

Objective meaning (up to the dash): Goldratt’s chain idea, which is a major part of your post, is limited.

Social meaning (up to the dash): I’ve surpassed Goldratt and can look down on his stuff as limited. You’re a naive Goldratt newbie who is accepting whatever he says instead of going beyond Goldratt. Also calling chains a “metaphor” instead of “model” is a subtle attack to lower status. Metaphors aren’t heavyweight rationality (while models are, and it actually is a model). Also Dagon is implying that I failed to recognize limits that I should have recognized.

Objective meaning continued: There’s some sort of attempt at an argument here but it doesn’t actually make sense. Saying there is variance in two places is not a limitation of the chain model.

Social meaning continued: saying a bunch of overly wordy stuff that looks technical is bluffing and pretending he’s arguing seriously. Most people won’t know the difference.

Most real-world situations are more of a mesh or a circuit than a linear chain, and the analysis of bottlenecks and risks is a fun multidimensional calculation of forces applies and propagated through multiple links.

Objective meaning: Chains are wrong in most real world situations because those situations are meshes or circuits [both terms undefined]. No details are given about how he knows what’s common in real world situations. And he’s contradicting Goldratt who actually did argue his case and know math. (I also know more than enough math so far and Dagon never continued with enough substance to potentially strain either of our math skills sets).

Social meaning: I have fun doing multidimensional calculations. I’m better than you. If you knew math so well that it’s a fun game to you, maybe you could keep up with me. But if you could do that, you wouldn’t have written the post you wrote.

It’s screwy how Dagon presents himself as a Goldratt superfan expert and then immediately attacks Goldratt’s ideas.

Note: Dagon stopped replying without explanation shortly after this, even though he’d said how super interested in Goldratt stuff he is.

Donald Hobson

I think that ideas can have a bottleneck effect, but that isn't the only effect. Some ideas have disjunctive justifications.

Objective meaning: bottlenecks come up sometimes but not always. [No arguments about how often they come up, how important they are, etc.]

Social meaning: You neglected disjunctions and didn’t see the whole picture. I often run into people who don’t know fancy concepts like “disjunction”.

Note: Disjunction just means “or” and isn’t something that Goldratt or I had failed to consider.

Hobson then follows up with some math, socially implying that the problem is I’m not technical enough and if only I knew some math I’d have reached different conclusions. He postures about how clever he is and brings up resistors and science as brags.

I responded, including with math, and then Hobson did not respond.

TAG

What does that even mean?

Objective meaning: I don’t understand what you wrote.

Social meaning: You’re not making sense.

He did give more info about what his question was after this. But he led with this, on purpose. The “even” is a social attack – that word isn’t there to help with any objective meaning. It’s there to socially communicate that I’m surprisingly incoherent. It’d be a subtle social attack even without the “even”. He didn’t respond when I answered his question.

abramdemski

There is another case which your argument neglects, which can make weakest-link reasoning highly inaccurate, and which is less of a special case than a tie in link-strength.

Objective meaning: The argument in the OP is incomplete.

Social meaning: You missed something huge, which is not a special case, so your reasoning is highly inaccurate.

The way you are reasoning about systems of interconnected ideas is conjunctive: every individual thing needs to be true.

Objective meaning: Chain links have an “and” relationship.

Social meaning: You lack a basic understanding of the stuff you just said, so I’ll have to start really basic to try to educate you.

But some things are disjunctive: some one thing needs to be true.

Objective meaning: “or” exists. [no statement yet about how this is relevant]

Social meaning: You’re wrong because you’re an ignorant novice.

(Of course there are even more exotic logical connectives, such as implication or XOR, which are also used in everyday reasoning. But for now it will do to consider only conjunction and disjunction.)

Objective meaning: Other logic operators exist [no statement yet about how this is relevant].

Social meaning: I know about this like XOR, but you’re a beginner who doesn’t. I’ll let you save face a little by calling it “exotic”, but actually, in the eyes of everyone knowledgeable here, I’m insulting you by suggesting that for you XOR is exotic.

Note: He’s wrong, I know what XOR is (let alone OR). So did Goldratt. XOR is actually easy for me, and I’ve used it a lot and done much more advanced things too. He assumed I didn’t in order to socially attack me. He didn’t have adequate evidence to reach the conclusion that he reached; but by reaching it and speaking condescendingly, he implied that there was adequate evidence to judge me as an ignorant fool.

Perhaps the excess accuracy in probability theory makes it more powerful than necessary to do its job? Perhaps this helps it deal with variance? Perhaps it helps the idea apply for other jobs than the one it was meant for?

Objective meaning: Bringing up possibilities he thinks are worth considering.

Social meaning: Flaming me with some rather thin plausible deniability.

I skipped the middle of his post btw, which had other bad stuff.

johnswentworth

I really like what this post is trying to do. The idea is a valuable one. But this explanation could use some work - not just because inferential distances are large, but because the presentation itself is too abstract to clearly communicate the intended point. In particular, I'd strongly recommend walking through at least 2-3 concrete examples of bottlenecks in ideas.

This is an apparently friendly reply but he was lying. I wrote examples but he wouldn’t speak again.

There are hints in this text that he actually dislikes me and is being condescending, and that the praise in the first two sentences is fake. You can see some condescension in the post, e.g. in how he sets himself up like a mentor telling me what to do (and note the unnecessary “strongly” before “recommend”. And how does he know the idea is valuable when it’s not clearly communicated? And his denial re inferential distance is actually both unreasonable and aggressive. The “too abstract” and “could use some work” are also social attacks, and the “at least 2-3” is a social attack (it means do a lot) with a confused objective meaning (if you’re saying do >= X, why specify X as a range? you only need one number.)

The objective world meaning is roughly that he’s helping with some presentation and communication issues and wants a discussion of the great ideas. But it turns out, as we see from his following behavior, that wasn’t true. (Probably. Maybe he didn’t follow up for some other reason like he died of COVID. Well not that because you can check his posting history and see he’s still posting in other topics. But maybe he has Alzheimer’s and he forgot, and he knows that’s a risk so he keeps notes about stuff he wants to follow up on, but he had an iCloud syncing error and the note got deleted without him realizing it. There are other stories that I don’t have enough information to rule out, but I do have broad societal information about them being uncommon, and there are patterns across the behavior of many people.)

MakoYass

I posted in comments on different Less Wrong thread:

curi:

Are you interested in extended discussion about this, with a goal of reaching some conclusions about CR/LW differences, or do you know anyone who is?

MakoYass:

I am evidently interested in discussing it, but I am probably not the best person for it.

Objective meaning: I am interested. My answer to your question is “yes”. I have agreed to try to have a discussion, if you want to. However, be warned that I’m not very good at this.

Social meaning: The answer to your question is “no”. I won’t discuss with you. However, I’m not OK with being declared uninterested in this topic. I love this topic. How dare you even question my interest when you have evidence (“evidently”) that I am interested, which consists of me having posted about it. I’d have been dumb to post about something I’m not interested in, and you were an asshole to suggest I might be dumb like that.

Actual result: I replied in a friendly, accessible way attempting to begin a conversation, but he did not respond.

Concluding Thoughts

Conversations don’t go well when a substantial portion of what people say has a hostile (or even just significantly different) social (double) meaning.

It’s much worse when the social meaning is the primary thing people are talking about, as in all the LW replies I got above. It’s hard to get discussions where the objective meanings are more emphasized than the social ones. And all the replies I quoted re my Chains and Bottlenecks post were top level replies to my impersonal article. I hadn’t said anything to personally offend any of those people, but they all responded with social nastiness. (Those were all the top level replies. There were no decent ones.) Also it was my first post back after 3 years, so this wasn’t carrying over from prior discussion (afaik – possibly some of them were around years ago and remembered me. I know some people do remember me but they mentioned it. Actually TAG said later, elsewhere, to someone else, that he knew about me from being on unspecified Critical Rationalist forums in the past).

Even if you’re aware of social meanings, there are important objective meanings which are quite hard to say without getting offensive social meaning. This comes up with talking about errors people make, especially ones that reveal significant weaknesses in their knowledge. Talking objectively about methodology errors and what to do about them can also be highly offensive socially. Also objective, argued judgments of how good things are can be socially offensive, even if correct (actually it’s often worse if it’s correct and high quality – the harder to plausibly argue back, the worse it can be for the guy who’s wrong).

The main point was to give examples of how the same sentence can be read with an objective and a social meaning. This is what most discussions on rationalist forums where explicit knowledge of social status hierarchies is common look like to me. It comes up a fair amount on my own forums too (less often than at LW, but it’s a pretty big problem IMO).

Note: The examples in this post are not representative of the full spectrum of social behaviors. One of the many things missing is needy/chasing/reactive behavior where people signal their own low social status (low relative to the person they’re trying to please). Also, I could go into more detail on any particular example people want to discuss (this post isn’t meant as giving all the info/analysis, it’s a hybrid between some summary and some detail).

---

Update: Adding (on same day as original) a few things I forgot to say.

Audiences pick up on some of the social meanings (which ones, and how they see them, varies by person). They see you answer and not answer things. They think some should be answered and some are ignorable. They take some things as social answers that aren’t intended to be. They sometimes ignore literal/objective meanings of things. They judge. It affects audience reactions. And the perception of audience reactions affects what the actual participants do and say (including when they stop talking without explanation).

The people quoted could do less social meaning. They’re all amplifying the social. There’s some design there; it’s not an accident. It’s not that hard to be less social. But even if you try, it’s very hard to avoid any problematic social meanings, especially when you consider that different audience members will read stuff differently, according to different background knowledge, different assumptions about context, different misreadings and skipped words, etc.

You can post messages (comments) here to test whether they're formatted how you expect. Messages here won't appear on the Recent Messages page or in the Messages RSS feed.

Discussion info link.

In the sci-fi novel Ring, Stephen Baxter wrote (my emphasis):

Lieserl had learned about the Qax. The Qax had originated as clusters of turbulent cells in the seas of a young planet. Because there were so few of them the Qax weren't naturally warlike -- individual life was far too precious to them. They were natural traders; the Qax worked with each other like independent corporations, in perfect competition. They had occupied Earth simply because it was so easy -- because they could. The only law governing the squabbling junior races of the Galaxy was, Lieserl realized, the iron rule of economics. The Qax enslaved mankind simply because it was an economically valid proposition. They had to learn the techniques of oppression from humans themselves. Fortunately for the Qax, human history wasn't short of object lessons.

This is explicitly claiming that slavery is economically beneficial, just like the 1619 project claims. There's something awful about how many people are fans of slavery. Immoral but practical!? No, it's immoral and impractical. Free trade is better at creating wealth. These people never read Mises or offer actual economic arguments.

People produce more (and do better thinking, which leads to e.g. scientific progress) when they aren't slaves or oppressed. And free trade avoids wasting resources on task masters, whips, chains, lasting resentment decades later, etc.

Baxter, by contrast, thinks that slavery is so great that it's worth learning how to oppress people in order to enslave them, even though all you knew how to do before was trade, and you'd already gotten very rich and powerful by trade.

BTW, in the story, the Qax ended up much worse off as a result of oppressing humanity. Lieserl is reviewing history and knows that outcome. Baxter's own story is a counter-example to his claim.

PS I liked the book. This was just one minor part.

I made eight videos about discussing social dynamics and conversation with gigahurt. In the videos, I explain to InternetRules how to make an idea tree and we make it.

Playlist of videos.

This is the idea tree for the conversation:

You can click the image to expand or view the PDF.

And here's more info about making idea trees.

Well-Kept Gardens Die By Pacifism by Eliezer Yudkowsky:

Good online communities die primarily by refusing to defend themselves.

Somewhere in the vastness of the Internet, it is happening even now. It was once a well-kept garden of intelligent discussion, where knowledgeable and interested folk came, attracted by the high quality of speech they saw ongoing. But into this garden comes a fool, and the level of discussion drops a little—or more than a little, if the fool is very prolific in their posting. (It is worse if the fool is just articulate enough that the former inhabitants of the garden feel obliged to respond, and correct misapprehensions—for then the fool dominates conversations.)

And what if you’re wrong about who the fools are?

What if you’re biased?

Where are the error correction mechanisms? Where are the objective tests to check that you aren’t screwing up?

Where’s the transparency? The rule of law, not rule of man? Where are the checks and balances?

If all you’ve got (as the moderator) is “I trust my judgment” then you’re just like everyone else, including the fool.

If you add some methods to try to have good judgment and try not to be biased … that’s not enough. Trying really hard, but ultimately trusting yourself and just going by what makes sense to you … is not a good enough defense against bias. Bias is powerful enough to beat that.

If fundamentally you’re just assuming your biases are pretty small and manageable, you are a fool. We are all alike in our infinite ignorance. We’re at the beginning of infinity. We’re so lost and confused in so many ways. We have big errors. Really big errors. Some are society-wide. And when we suppress everything that seems really quite wrong to us, what we’re doing is suppressing outliers. Yeah negative outliers are more common than positive outliers. It’s easier to be wrong than right. But how do you avoid suppressing positive outliers?

There are mechanisms you can put in place to make it harder to act on whim, bias, irrationality, etc.

E.g. you can explain why you take moderator actions. And you can field questions about your actions and reply to criticism. You could reply to all such things, plus followups. If you won’t do that, that’s a cutoff where you’re blocking error correction. And what have you done to prevent this cutoff from protecting the biases you may have?

Yes defense mechanisms are needed. But why can’t they be efficient and reusable ways to address the arguments of everyone, including fools? And a resilient forum where people think for themselves about what to focus attention on. If people want curation, fine, no problem, post curated selections somewhere – and leave the open forum also in existence. Post weekly favorites or whatever for the readers who don’t want to find the good stuff themselves. The curated view on the forum doesn’t have to be the only view. You can have an open public space, and a walled garden, both. Or dozens of competing walled gardens with different curators (thought only a few would probably have the vast majority of the popularity). But that’s dangerous. The curators may be biased. They may curate mostly by social status, for example. They may not know they do that. They may not understand some of their own systematic biases.

You have to fundamentally stop assuming you’re right or probably right and take seriously that you need error correction mechanisms to keep yourself honest. You can’t trust your own integrity. Don’t bet your life or your forum on your own integrity.

Scientists don’t bet science on their own integrity. Integrity helps. Try to have it. But science isn’t like “ok well do your best with the experiment and if you have integrity it should work out ok”. Instead experiments are designed to work out ok even if the experimenters can’t be trusted. The mechanisms don’t have unlimited resilience. Egregious scientific fraud has to get caught by outsiders. The scientific method makes that easier to catch. It’s harder to fake your experiments when there are procedures to follow, documentation to provide, etc. Most people are trying to cheat without full conscious intention though, so that’s easier to deal with. Having a little bit of integrity is really helpful compared to none. And anti-bias mechanisms with transparency and stuff do put a leash on the bad faith cheaters.

Double blind makes it harder to cheat even if you want to. You can be really biased, and your bias can control you, but if you follow the rules of double blind then it’s much harder for you to bias the results.

Control groups don’t care if you’re biased. That’s a system which is hard to cheat without extreme blatantness like skipping the control group entirely and using fabricated data. And it’s hard to do that because of transparency. And even if you get away with it, your results won’t replicate.

You’re expected to write about sources of error. If you don’t, people will write them for you and be more wary of your claims. If you do, people will consider how severe the issues are. If you write them but try to bias them, you’ll have a harder time convincing people who don’t share your biases. And when you leave stuff out, people can notice and then it’s harder for you to answer critics by claiming you totally knew about that and took it into account already.

Even when everyone shares biases, methods like “make a hypotheses. write it down. plan an experiment to test it. write down what results will agree with or disagree with the hypotheses. test it. compare the results to the predictions.” are capable of correcting everyone. That kind of method makes it way harder to fool yourself. If you skip steps like writing things down as you go along, then it’s much easier to fool yourself.

Forums need the same thing. Write down rules in advance. Make the rules totally predictable so people can know in advance what violates them or not. Don’t rely on potentially-biased judgment.

And when you must use judgment at a forum, be like a damn scientist, publish results, talk about sources of error, answer your critics, answer all the questions and doubts, etc. Take that seriously. Discuss the matter to conclusion. At least don’t be and stay wrong if anyone in your community knows you’re wrong and will explain it.

If you can’t resolve discussions, fix that and consider that maybe you’re a bit of a fool. If you don’t know how to get to conclusions with your critics, or manage those discussions without just ignoring arguments without answering, you need better systems and shouldn’t be so high and mighty to proclaim who is a fool.

Ideally, let the fool defend himself, too, Don’t just let others defend him. Topic-limit him during that discussion if you must so he can’t participate in other discussions until it’s resolved.

---

Also in the article, EY says academia is a walled garden that keeps the fools out, so that’s why ppl are naive and don’t realize they need censors. And I’m like: Yeah that is exactly what academia is and it’s fucking awful there. And academia’s walls are 99% based on social status.

What is your forum doing to prevent social status bias from deciding who the fools are? What explicit policies does it have that could actually work in case that you, the moderators, are social status biased?

EY’s answer is basically “if the mods suck, the forum is fucked”. Just find other, better people to rule. What an awful answer. Seriously that’s his position:

Any community that really needs to question its moderators, that really seriously has abusive moderators, is probably not worth saving.

No! You need good systems, not sinless, unbiased moderators (nor mediocre moderators who aren’t all that bad and you just put up with their errors). It’s like: dear God we’re not going to get unbiased politicians; we need a government system that works anyway. Forums are the same thing. Write out laws in advance. Make new laws with a process. Anyone who doesn’t violate a law in a crystal clear way gets away with it. etc. Otherwise the moderators will have some sort of biases – doesn’t everyone? – and they’re going to oppress the people with other biases who are such a source of intellectual diversity. Broaden your horizons instead of getting rid of all your dissidents and misfits and outliers. God damn you, “overcoming bias” you say?

We know a lot about how to deal with bias, abuse, unfairness, etc. from our legal system and from science. Then people don’t apply those lessons when they have power over a forum.

I have seen rationalist communities die because they trusted their moderators too little.

…

Here—you must trust yourselves.

You don’t overcome your biases by trusting yourself.

Don’t be a skeptic. Don’t never reach any conclusions or judgments. Don’t be scared to act in your life. But don’t trust your moderation to be unbiased. Have layers and layers of safety valves that provide different opportunities for you to be corrected if you’re wrong or biased. Never trust yourself such that you don’t think you need any mechanisms to keep you honest.

---

The scientific method makes it harder to be wrong and stay wrong. We need stuff like that for running forums too.

The scientific method does not consider it adequate for scientists to read about how to be good scientists, think about it, discuss it, do their best, and trust themselves. That would be ridiculous.

And the laws have lots of protection against the errors and biases of the individuals who enforce them. E.g. police, judge, jury and executioner are all different people. At forums they’re usually all the same people, which means you need even more safety mechanisms of other types. And we write out clear, objective laws in advance – no retroactive crimes. And there are appeals. And defendants have rights and privileges that the judges have to respect, and that really is enforced in various ways. And we try to limit the discretion of judges to be biased or play favorites by making the law really clear about what they should do. We don’t do that perfectly but we try and it helps. And when we make laws, they are (or at least should be) pretty generic and based on general principles instead of targeting specific individuals or small groups – we want our laws to be relevant for future generations and different societal situations, rather than overly specific to deal some current events.

Highlights from the Blackmail Debate (Robin Hanson vs Zvi Mowshowitz) by Ben Pace:

Here's my overview of their positions.

• Zvi thinks that blackmail would incentivize a whole host of terrible actions, such as trying to trick people into norm violation, and people becoming intensely secretive even around their closest friends and family.
• Robin thinks that blackmail is a weird rule, where you cannot ask for money to keep a secret, but the other person is allowed to offer it (e.g. people can offer you money if you sign an NDAs). This makes no sense and Robin is looking for any clear reason why making one side of this deal should be illegal.

Thoughts Before Reading More of the Post

there are practical problems. e.g. blackmailers may make additional demands later, and it’s hard to hold them to the original deal b/c if you bring up the deal for enforcement then your secret comes out

and ppl actively trying to get blackmail material is a problem, as Zvi says

NDAs are done preemptively which is different. you ask someone to agree to keep X secret as a condition of sharing X with him.

blackmail is someone who already knows X in advance and is now negotiating for potential secrecy.

NDAs are commonly signed with no payment, or at least no direct payment for them (required with employment)

ppl sign NDAs b/c they want access to the info b/c they have an actual productive purpose for using that info themselves – often a business purpose like providing consulting advice about it or developing software related to it.

blackmailers commonly have no productive uses of the info.
their choices are roughly do nothing or hurt someone.

in cases where the blackmailer got the info legally and it isn’t e.g. copyrighted, and he isn’t lying (no fraud), it’s unclear where the initiation of force is if he shares it with the world. and if that’s not force, it’s hard to see how offering not to do it for money would be force.

it’s easy to see why this sort of thing is bad for society, at least in the sorts of scenarios typically associated with blackmail.

let’s consider an example. i see you in public with a woman. you kiss. i know you’re married. i don’t take any photos. i just witness it and then tell you that i’m going to (accurately and honestly) tell your wife unless you pay me $1000. if you pay once, i contractually agree to never use this again, even if I see you with that woman again. (i can ask for more money if i catch you with a different woman, though.)

i tried to keep it simple by having everything occur in public and avoiding photos/recordings. and i wouldn’t exaggerate to your wife, and you don’t think i would, so there isn’t an issue of fraud (tricking her, or you getting the impression that i’m threatening to say anything untrue or misleading to her).

damages ought to be big – much more than $1000 – if i tell your wife anyway or carelessly tell other people who spread the news. this can be a practical problem if i’m not wealthy enough to recover significant damages from. blackmailers are often hard to sue if they break contract. but in terms of principles, the contract excludes asking for additional payments or screwing you over anyway, and if i do that i’m now clearly guilty of initiating force.

why is this such a problem? because you’re doing something you don’t want your wife to know about. you’re lying to your wife and you want to get away with it. maybe we should blame you for the problematic situation, not me. if i keep my mouth shut, we still don’t get an OK situation. getting a good situation requires you changing your actions.

i think that’s a known idea. roughly: don’t do stuff that you could be blackmailed for.

in other words, if it’s worth $10,000 to you for no one to know you’re cheating on your wife, then don’t kiss other women in public, unless you don’t mind paying. (and if 50 people see you, and realize what’s going on, then you’re not going to be able to pay enough money to them all while staying under the limit of what the secret is worth to you; you’ll just have to accept you’re caught).

what’s hard about avoiding blackmailable actions?

there are lots of reasonable choices which society disapproves of. desire for privacy is often ok. this can happen even if you don’t lie to anyone. a simple example is you’re gay and don’t want people to know, though that generally involves some lying.

some things look bad but are ok if explained. but if the public is told, you won’t get a chance to immediately explain to everyone as soon as they get the info. e.g. maybe there is a photo of you with a young child. people might believe you’re a pedophile when actually you’re volunteering for the Big Brother charity. the photo is misleading because you were roleplaying an abduction scenario to help educate him. the photographer doesn’t even known, but instead of telling the public he goes to ask you for money … wait that’s not ok, he shouldn’t be asking for money to help cover up what he believes to be a crime. so let’s make the scenario milder, you’re just hugging him … well either that’s innocent or evidence of a crime so from either pov he shouldn’t be asking you for money (either he thinks it was fine so what is his threat exactly … to mislead the public? – or he thinks it suggests a crime so he shouldn’t cover it up).

There’s no positive duty to report crimes, but getting into a financial relationship to hide a crime (or what you think may be a crime) sounds rightly illegal. That’s like being hired by the criminal as a sort of accomplice. Note: It’s still bad if you’re paid with favors (services) instead of money.

OK so I want a scenario where it’s merely embarrassing, so being paid to help you hide it is ok. And the person being blackmailed hasn’t lied to anyone or done anything that’s actually bad. So I misjudge you as being a friend and tell you about my BDSM sex with my wife which I don’t want the public to know about. I didn’t think you were contractually bound not to tell; I just thought you were nice. I haven’t lied to the public about my sex life; I’ve just kept it private.

So you say you’ll post it on your blog unless I give you $100. You’d be doing this for the sole purpose of hurting me. But it seems to be within your rights to post it. And if it is, what’s wrong with offering a trade where you don’t do it in exchange for $100?

Normally trade is value for value. It’s supposed to be mutual benefit. Refraining from hurting me isn’t a service in the usual sense.

We could try to compare it to: I offer my neighbors $100 not to play music this weekend (it’s fine with headphones, I just don’t want my party disrupted). So now i’m paying for them not to do something. Why? Well they often play music with speakers. It’s something they gain value from. I want them to give it up so I compensate them.

We can imagine the reverse scenario in two ways. First, the neighbors come to me and say “I heard you have a party this weekend. We know our music often annoys you. We’ll be quiet all weekend for $100 if you want. Otherwise we’ll act normally.” Now compare this to scenario 2: The neighbors are normally quiet, hear about the party, and come say they will be louder than usual, just to disrupt your party, unless you pay $100 for quiet.

I think this is important. Paying someone to change their usual behavior and refrain from something is fine. The source of that behavior was to live their own life, not to hurt you. But if they are threatening to do a behavior for the purpose of hurting you, not to improve their own life, then something bad is going on. This is kinda hard for law to deal with though – it’s hard for courts to judge they aren’t playing loud music as a new hobby or whatever – though it becomes easier to judge if they approach you with the threat to play loud music unless you pay.

The purpose of political rights is to enable you to live your own life and have a good life. It’s not intended to give you any right to hurt others for no benefit to yourself (or to threaten to do that in order to demand money).

So I should reexamine first principles. Basically I have a right to control my body and, by extension, my property. I have freedom to live my life. But I don’t get to interfere with others. Interfering with other people’s body or property is initiation of force. In some cases, it’s hard to tell what counts as interference. Light and sound from my activities on my land often find their way onto my neighbor’s land and could annoy him (or could leak information that I don’t want him to have – and my first guess is I should need to take only reasonable precautions, and if he buys super high tech spying equipment then he’s doing something wrong despite only using it on the physical particles on his property). Tradition and law have to deal with stuff like sharing rivers, noise pollution, adding a tall building on my property that could block the view from your house, ugly stuff on my property being visible from your property, etc.

Part of how we deal with these things is by good faith and good will, not just by strictly defining what is force. It’s too hard to get along purely by saying what is force or not. That’s not enough to build up a good society. You need people who want to get along and mostly try to avoid conflict. Blackmail violates that.

Since blackmail is unnecessary to pursuing positive values in your own life, and undermines and attacks social harmony, I think it’s reasonable to make it illegal. This may change in the future when everything in public is getting recorded and posted online automatically and you’ll just need to be more careful in public or figure out some other way to deal with the situation, but for now it’s good that people generally don’t have to be too scared while in public (or while on their own property but inadvertently sending quite a bit of information (information from a physics perspective) off their property because they lack adequate technology to control that).

I do think we should be very careful about legitimizing laws that go beyond prohibiting force (including fraud and threat). The main danger of additional laws is they can restrict people’s freedom to have a good life, which this one doesn’t because I think the defining feature of blackmail is threatening to do something that isn’t part of pursuing your own values, it’s something you’d do just to hurt others. Laws against malice and tricky and dangerous in general but this looks OK to me. Trade should be for mutual benefit, not for “i found a way to harm you that doesn’t count as force; pay me not to do it”.

But fundamentally rights have a positive purpose to enable a good life and to resolve conflicts peacefully. The fundamental thing isn’t to ban certain actions like force. That’s a means to an end. So if someone finds a way to hurt people that isn’t banned (and IRL it is banned, but it isn’t clearly banned by the non-aggression principle), that looks to me like finding a hole in the rules while going against the purpose of the rules. Whereas if I thought the NAP (non-aggression principle, aka don’t initiate force) was primary, then I might think “technically doesn’t violate NAP, therefore good”. But I don’t see the NAP as a source of values or morality itself, directly.

I do think this merits more thought and analysis. I’m not fully satisfied yet. But I might be done anyway. This isn’t my main field and I have other things to do. Now I’ll read the debate highlights and see if I have any reason to change my mind already.

Thoughts After Reading the Rest of the Post

Hanson says legalizing blackmail could reduce affairs. So I guess the positive value of blackmail, in some cases, is trying to clean up society. I want to out people who cheat on their spouses to make a better world. That’s part of pursuing my values.

But if that’s really what I want, I can just do it. I wouldn’t be like “pay me to let you keep cheating on your wife”. I’d just get the info and share it. I’d try to get paid by the tabloids or by people who also value cleaning up the sexual morals of society.

So Hanson’s first claim, under “What's good about blackmail”, hasn’t changed my mind/analysis.

Zvi Mowshowitz: I believe that most of the time, most people are not, when they seek information, primarily looking to harm someone. They're primarily looking to benefit themselves, or benefit their friends or their allies in some way. Hurting someone else is mostly a side effect.

Robin Hanson: That's also true with blackmail. The main effect is the money, not the hurt. Their main motivation is to get the compensation.

This part stood out to me because it’s similar to some stuff I was talking about. I think seeking out and publishing info to pursue your own values is fine, but I take issue with doing it in order to threaten to harm someone. If you really want the info because you value it, try to get paid by other people who value it, rather than getting paid to hide the info that you value being public.

At the end they poll the audience re changing their mind. What about me? My initial opinion was blackmail sounds bad but I’m not that sure because it’s not clear how it initiates force. My updated opinion is that, due to my own analysis, I’m more confident that blackmail should be (stay) illegal. But if it came up in my life I’d want to do more analysis before acting (unless I didn’t have time to analyze, in which case I’d view blackmailing as bad and e.g. refrain from blackmailing someone even if I had no fear of getting in legal trouble for doing it).

---

View this post at Less Wrong.