Merging With An LLM: Can an LLM reconstruct my values from small data?

LLM capabilities are increasing. They create risks for financial and logistical infrastructure, and make engineered pandemics more likely. Work on model safety and societal mitigation efforts follows. Perhaps even a pause enters the discussion.

But this post addresses an emotional part of human disempowerment that comes with smarter AI systems. Computers can do things better than we can, and this hurts the ego. It sucks that we can no longer be uniquely good at something. When humans were the most intelligent things around, if someone was unskilled, that person could at least dream that one day greatness was possible. But now, even the most skilled of us are unlikely to win against the machines on any timescale. Scientists and mathematicians feel demoralized. So do artists.

This feels hurtful, partly because we consider computers separate from us. The computer is a different thing, separate from our sense of self. This separate entity removes a source of happiness, like the feeling of being good at something. This makes us feel bad and disempowered. Another word for this is competition. We are in competition with the machines, and we have lost. The language we use to describe this is "Y has beaten me at X."

But could the framing change so that it feels better? Could reframing make space for excitement about LLM intelligence? If so, perhaps feeling more empowered follows. One language game might consider LLMs as part of our being. Right now, an ideological boundary exists between LLMs and ourselves. Could the feeling of disempowerment be solved by removing this ideological line?

Why do English football fans, many of which are bitter rivals during the English Premier League, suddenly all band together during the World Cup? They consider themselves as one entity, and remove the ideological lines from local teams during World Cup years. Most people in America felt good when the first moon landing happened. Among Americans, there was no competition nor adversity, despite Republican and Democrat divides. Why? American humans considered themselves as one entity. For a moment, America was Neil Armstrong (except the Soviets, who considered themselves as separate).

Suppose that the ideological boundary between me and an LLM gets removed. In this paradigm, the LLM becomes an extension of me. I merge with the LLM. If I do this, will I feel less bad about my inadequacy against the machine? How can I compete with something that I consider part of myself?

Maybe this is the solution to the disempowerment. If I prompt an LLM to do something great, then the emotional win is for both the LLM and me. I cease to feel bad.

One complication exists. The ideological merge almost surely involves a third party, an unwanted dinner guest. This third party is the model provider; OpenAI, Anthropic, Deepmind, or whoever. With a third party, many bad things can intrude in the merge. They might inject their values in unsavory ways. They could invade privacy. The LLM might run away and imitate in ways that are unwanted.

Therefore, the only acceptable merge would be merge with a local LLM on a local GPU cluster. This local LLM would reflect my values and beliefs and enhance them in ways that feel comfortable. Right now, no such service exists for me to design my own local aligned LLM. I do not have enough compute. I have been experimenting with models that can run on Mac M4 Studio memory but the results have been bad.

I need a way to fully specify my values and beliefs, and transfer them fully to the local LLM. One way could be having the LLM learn continuously from my Obsidian and blog, and then have it ask me about my beliefs to fill in the gaps. For unspecified moral problems, maybe it could extrapolate from what I have already.

I do not know how much data it would need though. This is because mathematically, my ideals and beliefs come from a very high dimensional space. The high dimensional space changes, and it is unclear what the principal eigenvectors of this high dimensional space are. The high dimensional space gets projected down to my writing, and this writing is supposed to reflect my values.

For an LLM to be fully aligned, it would need to take the low dimensional space of my writing and then recreate the high dimensional space that is my complete value set. But this is a problem of information loss. The idea of recreating a high dimensional space from low dimensional space has deep connections with complex systems and theoretical physics. The holographic principle in string theory relates information in lower dimensional boundaries to a higher dimensional bulk. In complex systems research, people try to recreate dynamics from a reduced representation called phase space.

But it is unclear how successful this approach would be in an LLM and values context. For example, would my feelings on capitalism be enough for an LLM to infer my feelings on abortion? Would my feelings on veganism be enough for an LLM to infer whether I want to buy that new toaster? Is this problem just a facade of the alignment problem that everyone is working on right now? It is all unclear.

Acknowledgements: Ideas sparked from an Independent Science Society dinner with Ariel Cheng. All views and mistakes are my own.