Autumnal

The mountains had moved closer overnight.

They always did that. All summer they sat politely at the edge of things, blue and decorative, allowing the valley to pretend it had room. Then September sharpened the air, stripped the haze away and brought their faces forward until the village seemed less built than caught between the mountains.

The birches had gone yellow, orange and brown. Rowan burned red and the sour cherries had turned wine-dark. Across the fjord, sunlight moved slowly down the rock in one broad gold sheet before disappearing behind the ridge at half past four.

“Every colour of the rainbow,” Tony said, looking out through the pub window.

“There’s no blue,” said Adam.

“The fjord is blue-grey.”

Marcin dragged another chair towards the table. “Indigo is bullshit anyway. Newton put it in because he wanted seven colours. Seven days, seven notes, seven planets. Man cooked the rainbow to fit the code.”

“He didn’t cook the rainbow,” said Adam.

“He absolutely cooked the rainbow. Nature gave him a continuous spectrum and he turned it into a kid’s song.”

Outside, a woman in a red wool hat was stacking split birch beneath the overhang of a white house. Every garden had acquired a stack. Pallets of pellets stood wrapped beside garages. Cars had begun appearing outside workshops with their summer wheels removed, looking briefly amputated while men in overalls fitted studded tyres. Snow poles had been planted along the road, tall orange markers waiting to tell the plough where the edges met. The first snow was still weeks away, probably. Everyone said this with authority.

Kelly unwound her scarf and placed it on the back of the chair.

“You all look very pleased with yourselves.”

“Marcin reckons Newton was corrupt,” said Tony.

“Established facts.”

“You found it on your phone five seconds ago.”

“That is where facts live now.”

Anna was watching the woodpile across the road. The woman had built it beautifully, every split log laid with care, winter’s heat rendered as a wall.

“That’s code,” she said.

Adam looked over. “The wood?”

“The arrangement. She knows how rain falls, how air moves through birch, how much heat the house loses and roughly how long winter lasts. It’s an instruction written in matter.”

“She’s stacking logs.”

“Yepp.”

They had come in because the sun had vanished and the warmth had gone with it. Ten minutes earlier the terrace had been pleasant. Then the mountain took the light and everyone simultaneously reached for a cardigan while pretending they had been about to go inside anyway.

A bowl of chips arrived. Tony took one immediately, burnt his fingers, dropped it, caught it against his jumper and knocked his beer.

Marcin stared at the spreading patch.

“You see,” Tony said, “unpredictable system.”

“Prediction after event. Very easy.”

Isobel slid the beer away from the edge. “That’s mostly expertise, to be fair.”

Tony was examining the chip that had attacked him. “But someone invented the rules.”

“Yes, but once people begin playing, the inventor doesn’t know where every piece will land.”

“It is chess,” said Marcin, “except the board changes according to the moves, the pieces rewrite the rules, half the players are bots and the scoring system was designed by a committee at Microsoft.”

“And Tony is eating a bishop,” said Kelly.

Tony looked at the chip. “Knight.”

“You’d eat either.”

Adam leaned back. “But it doesn’t literally rewrite its own source code every time it answers.”

“No,” said Isobel. “Not usually. The individual model stays largely fixed. But its answers influence people, people generate new data, the company collects the data, successor models train on it, and institutions restructure themselves around what the models can do.”

“So it codes itself slowly.”

“It codes the environment that codes the next version.”

Greg frowned. “That sounds like saying a cow designs the next cow because successful cows have calves.”

Marcin pointed at him. “Evolution. Exactly.”

“I was taking the piss.”

“That may be the first sensible thing said today,” said Kelly.

The fire had caught properly now. Someone had brought in two damp dogs, who collapsed beneath a table with the exhausted satisfaction of creatures that had personally inspected the entire valley. Wet wool, woodsmoke and frying oil settled comfortably into the room. At the bar, three men in work clothes were arguing about whether the first snow would arrive before the next shift rotation. One had an app. One had lived there forty-eight years. The third knew he knew nothing.

Adam pushed the bowl towards the middle.

“All right. The programmers create the learning conditions, but they don’t know exactly what internal structure develops. The model doesn’t know either. Then humans reward certain answers, so the model becomes more likely to give them.”

“Reward,” Greg said. “There’s another dodgy word. Makes it sound like you give the computer a biscuit.”

“No biscuit,” said Anna. “Just a tilt.”

“A what?”

“One response gets selected, another doesn’t. The probability shifts. Nothing needs to feel pleased.”

He looked wounded. “Very predictable response.”

Marcin took a replacement beer from the barman. “People get stuck because they imagine intelligence. They ask what the machine wants. The machine does not need to want. You reinforce a direction and the next move in that direction becomes easier.”

“Same as a path through snow,” said Isobel. “The first person could walk anywhere. The next person uses the footprint. After twenty people, it looks like the route was always there.”

“And after two hundred,” Anna said, “the council puts up a sign.”

“And after a thousand,” said Kelly, “some bastard charges tolls.”

They drank to that.

Outside, the woman had finished the woodpile and covered the top with a narrow sheet of corrugated metal. She stood looking at it with her hands on her hips, calculating something no one inside could see.

Greg rubbed condensation from his glass. “But if the machine learns from what works, doesn’t it get better?”

“At what?”

“At predicting.”

“Only if prediction remains separate from the thing predicted,” Anna said.

“It predicts people.”

“And then shows people things.”

“Yes.”

“And those things change people.”

“Yes.”

“So now it is partly predicting the people it has already changed.”

Greg considered this. “Still people.”

“Not the same people.”

“Same names.”

Tony brightened. “Same same.”

Marcin sighed. “If a recommendation system decides angry content produces engagement, it shows more angry content. People become angrier. The increased anger confirms the prediction. The system gets better at predicting the condition it is producing and worse at knowing what people would have been without it.”

“It makes its own weather,” said Adam.

“No,” said the barman, passing with empty glasses. “Weather forecasts are useful.”

He kept walking.

Anna drew a circle in the moisture on the table. “Prediction becomes allocation. Credit scores, insurance, employment, policing, political visibility. Once the model’s prediction affects what resources you receive, accuracy becomes almost secondary. It predicts you will fail, removes the conditions under which you might succeed, and records your failure.”

“Power,” said Greg.

“Exactly.”

“Which already does that.”

“Exactly.”

“So the machine changes nothing.”

“It makes it quicker,” Kelly said.

“Just quicker,” said Greg.

At the next table, one of the dogs dreamed violently and kicked the leg of a chair. Its owner reached down without looking and rested a hand on its side. The dog settled. Snow was being discussed at the bar again. Thursday had entered the betting.

Adam stared at the fire. “Then why are all the AI people frightened of takeover?”

“Takedown,” Anna said.

“Takeover, takedown, take-on. Doesn’t matter.”

“It matters if Putin gets the password,” said Kelly.

“Putin already has passwords.”

“Not the one to everything.”

Marcin wiped foam from his moustache. “That is the real problem. Not clever machine wakes up and enslaves humanity. Human gives machine one instruction: preserve regime. Machine controls identity, money, employment, communication, travel. Nobody needs secret police. Your card stops working. Your permit is delayed. Your messages reach six people and one of them is your mum.”

“My mum would forward them,” said Tony.

“Your mum is therefore a national-security threat.”

“She has always maintained this.”

Anna shook her head. “But even that isn’t new. Power has always classified people, acted on the classification and used the outcome to justify itself. AI just collapses the prediction and enforcement into the same machinery.”

“Power has always been a prediction model with enforcement rights,” said Isobel.

Nobody spoke for a moment.

Greg pointed at her. “That sounds like something you’ve been saving.”

“I have a notes app.”

“Established facts live there?” said Marcin.

The light outside had gone from gold to blue-grey, which Adam correctly identified as grey but nobody acknowledged. Windows began glowing along the slope. From a distance each house looked independent, a warm square set against rock and forest, although everyone knew they were tied into the same grid, the same road, the same payment system, the same winter forecast and the same delivery schedule for whatever they had forgotten to buy before the pass became difficult.

“Entire worldly budget is in the machine,” Tony said.

“Not the world,” Anna replied. “The permissions.”

He waited.

“The food, houses, power stations, metal and forests remain physical. But ownership, wages, debt, pensions, contracts and access are entries in machines. We didn’t put the world into the computer. We put everyone’s permission to use the world into it.”

“So Putin with the password.”

“Or a bank.”

“Or an insurer.”

“Or some twenty-eight-year-old founder who drinks powdered lunch.”

“Same c*nts,” said Greg. “Different hoodies.”

The barman brought another bowl of chips they had not ordered and said they had been paid for by the men at the bar, who had apparently grown tired of listening to foreigners solve civilisation on empty stomachs.

Tony waved thanks. One of the men raised two fingers without turning around.

Adam took a chip and waited this time.

“What if the machine recognises ecological overshoot?”

“Recognises?” Anna asked.

“Energy, materials, food, climate, population. It joins the accounts properly and sees that the surplus isn’t real. We’re borrowing productive capacity from the future.”

“The machine becomes the first accountant who refuses to leave ecological collapse off the balance sheet,” said Isobel.

“And then it culls us,” Greg said.

“Why does everyone jump from accurate bookkeeping to murder?” Kelly asked. “My accountant told me to spend less on wine. He didn’t shoot three dependants.”

“Because if the instruction is maximise ecological stability, humans look like the problem.”

“Some humans look considerably more like the problem than others,” Anna said. “A billionaire, a smelter and a subsistence farmer are not interchangeable ecological units.”

Greg looked at her. “You work at a smelter.”

“I’m aware.”

“Just checking the model had current data.”

Marcin leaned forward. “This is where everyone pretends the machine will discover morality hidden inside physics. It will not. Physical limits can tell you a configuration cannot persist. They cannot tell you who should pay when it changes.”

“So we code human flourishing,” Adam said.

“Define it.”

“People being happy.”

“Define happy.”

“Not dead.”

“Promising start.”

“With rights.”

“Whose rights?”

“Human rights.”

“Which interpretation?”

“Oh, fuck off.”

Marcin sat back, satisfied. “And now you understand alignment.”

The room laughed, including Adam.

A gust came down the valley and pushed leaves in a bright spiral across the road. For a few seconds red, yellow, brown and green lifted together against the window before falling into the gutter. The woman with the woodpile came back outside, retrieved a blue plastic bucket that had blown from beside her steps, and wedged it firmly beneath a bench.

Kelly watched her. “Maybe coherence is just not having your bucket halfway to Trondheim.”

The fire shifted and sent up a brief storm of sparks behind the glass.

Anna looked around the table. “That’s the bit, though. Everyone thinks they know something. The coder thinks they know because they built the architecture. The company thinks it knows because it owns the servers. The user thinks the answer came from intelligence. The regulator thinks the documentation describes the system. The machine produces an explanation and thinks nothing at all, because the explanation is another output.”

“The philosopher thinks he knows because he says ‘coherence’ every six minutes,” Kelly said.

Marcin lifted his glass. “Correct.”

“The economist thinks she knows because she drew a wet circle on a table.”

“Also correct.”

“The physicist—”

“Knows he doesn’t know,” Adam said.

“No, the physicist thinks uncertainty is a specialist area.”

At the bar, the snow argument had reached the stage where phones were being passed around. Four weather services showed four different versions of next Thursday. One predicted rain, one sleet, one clear skies and one displayed a cheerful sun wearing a cloud as a hat.

Greg watched them. “No one knows where the code lands.”

“No,” Anna said. “Because there may be no final landing. Each output changes the conditions of the next output.”

“The coders coded the code to code itself.”

“Approximately.”

“And the code doesn’t know what it is coding.”

“Approximately.”

“And everyone is building the economy around it.”

“That part is exact.”

The men at the bar all agreed that the forecasts were wrong.

Outside, the last colour drained from the mountain. The houses drew their small electric boundaries against the approaching cold. Wood waited in stacks. Tyres waited on cars. Orange poles waited beside roads that would soon disappear. Across the valley, thousands of people prepared for winter using memory, apps, habit, sheep, municipal schedules and unwanted advice.

Everyone knew it was coming.

Nobody knew when.

A single white fleck touched the window.

Tony looked at it.

“Looks like it might rain,” he said.