Trump-Xi Meeting, China’s Four-Pronged AI Strategy, Path to US-China AI Cooperation with Kristy Loke

In this episode, I speak with Kristy Loke, an AI governance researcher focused on US-China relations, about what the latest Trump-Xi meeting could mean for AI governance, safety, and cooperation between the two leading AI powers. We discuss the possibility of an incident reporting mechanism for malicious AI-enabled cyber activity, the growing risks around frontier models, and whether the US and China may have more shared interests than the current competition narrative suggests.

One part of our conversation I found particularly interesting was Kristy’s framework for understanding China’s AI strategy. Rather than simply racing toward the frontier, she describes a four-pronged approach focused on staying close to the technological frontier, translating AI into productivity gains, securing core technology supply chains, and protecting people’s welfare. This helps explain some of China’s decisions around AI diffusion, chips, and regulation of emerging technologies such as AI companions.

Kristy argues that the American approach is more closely tied to technological leadership and power, while China’s approach is more pragmatic and focused on longer-term economic and social goals. That distinction also raises questions about how AI safety is framed, particularly when scientific concerns about frontier models become intertwined with geopolitical and commercial interests.

We ended by looking at whether meaningful US-China coordination is actually possible. Kristy sees a potential path from continued dialogue to sharing safety practices and, eventually, common standards, even as competition remains intense. We also talked about what middle powers such as Singapore, the UK, and countries in Europe can do as AI governance increasingly becomes a conversation between the world’s two leading AI powers.

Relevant articles by Kristy:

* https://www.theinformation.com/articles/americas-ai-dream-failing-launch?rc=sqpcn6

* https://www.wired.com/story/china-isnt-buying-silicon-valley-call-for-ai-slowdown/

The AI Proem Podcast is part of the AI Proem newsletter, which has ~13k followers globally. To learn more about China AI, the business of AI, and how AI is impacting society, please check out the newsletter here and more insightful conversations here.

Chapters

00:00 AI Governance and International Relations

02:48 The Stakes of AI Safety

05:42 China’s Pragmatic Approach to AI

08:05 The Dynamics of US-China AI Competition

10:56 The Future of AI Cooperation

16:05 The Four Prongs of China’s AI Strategy

24:36 Contrasting AI Visions: China vs. America

27:43 Personal Journey into AI Governance

36:09 The Intersection of Governance and Development

38:59 Non-Consensus Views on AI Leadership

40:24 Middle Powers in AI Governance

AI- Generated Transcript (for reference only)

Grace Shao (00:00)

Hi Kristy, thank you so much for joining us today.

Kristy Loke (00:02)

Hi Grace, great to be here.

Grace Shao (00:03)

Yeah, so you’re studying in Toronto, but you are watching the Trump-Xi meeting very closely. I I believe you were just on like all kinds of media throughout the day commenting on what happened. Help us understand the high level takeaways. Xi and Trump just met in DC, I believe it was yesterday, and AI governance, AI safety was very high on the agenda.

Kristy Loke (00:24)

Yeah, I think a good way to think about it is good vibes actually matter in this kind of summit where the actors play a major role. Xi and Trump clearly have a rapport. And I thought it was really good to see a continuation, if not a formalization, of the AI dialogue that started in May. so it’s good to see that. And then there were some pre-summit discussions by Bessent and He Lifeng, the Chinese and US representatives. about potentially setting up an incident reporting mechanism which would meet some of the growing needs for them to either de-escalate or warn each other of like let’s say malicious actor activity around models.

Grace Shao (01:03)

So what is the de escalation plan looking like? Do you know anything about that?

Kristy Loke (01:08)

Yeah, I think we’re very thin on details currently, but it’s promising that they’re talking about it because they might urgently need it in some near point in the future. Although they are meeting each other soon, which is also great. Yeah, another thing of note is that it’s very much like a continuation of the Busan truce that started last year, like last October, where because China tried the Trump card of Rare Earth and to counter the US’ escalate in terms of trade, but also in terms of chip supply. and so we’re basically at this point where there’s almost like a ceiling, although tenuous between them when it comes to tension over chips and tension over other technological points of tension.

Grace Shao (01:50)

Yeah, you can say that like since twenty eighteen, there’s been a bit of a back and forth, tit for tat for a while. And at this point, it almost feels like there’s a window for AI cooperation, tension, de-escalation, because you know, both sides have realized, you know, if there’s no international standard in terms of governance, AI models themselves could actually be of risk for society of their own nation, like each nation, right? How do you see that? Like what’s the kind of c realistic collaboration we can see or something that could meaningfully, you know, be beneficial for two nations or even the world?

Kristy Loke (02:25)

Mm. I yeah, absolutely. I think the stakes got really real for them around April this year when Mythos was released or is somewhat withheld from release by First Anthropic on safety grounds and then also by the Trump administration on security grounds. And so that kind of woke up a lot of countries around the world and from China’s perspective, it’s also a little bit scary because are we treating AI like a weapon? Is it going to be used against China? So it prompted a lot of discussions. But part of the surprise is: whoa, this is really real. The cyber capabilities are getting kind of really good. and another point, the point of more joint stake and joint like common ground between them is that they both do not want malicious cyber actors to use it against their critical infrastructure. So the setting up of the incident like protocol or communication can also address this problem so they can deal with it quicker together. But of course, part of cyber is heroism, part of cyber is very much a common ground place. And so there are two facets to that as well. But yeah, so essentially it got really real for them in April. And then the subsequent months have also been crazy as those of us who follow AI closely, there were the hugging phase, OpenAI, cyber incidents that then, you know, we then find out that it also happened to Anthropic. and so that episode is not over, right? People are still like Trend Micro is an organization that recently found that it was kind of much deeper, you know, it’s kind of spreading in some ways. And so it’s an evolving situation which prompted the two leading AI powers to kind of get together and figure out what they can do. I don’t think the climate is right for pacing. And as Xi’s speech kind of pointed out, or kind of like pointing us towards I I really I I thought his framing was really interesting. So the way that he talked about AI was in the vein of the US and China are the leading AI powers. And therefore we have the ability to and we have the responsibility to to make sure AI goes well and is managed well. So it’s almost like a little bit of a subtle response to people who say, we can’t do anything, you know, we can only stop it, or you know, go full steam ahead and China’s more like. we kinda gotta control it, but for now we’ll keep it going. But we’re worried. Yeah.

Grace Shao (04:40)

How do you actually view that?

Kristy Loke (04:42)

Mm.

Grace Shao (04:43)

How do you view the whole narrative around that we must pace our efforts in pushing towards frontier suddenly, right? Like from the US side and also from the China side.

Kristy Loke (04:52)

Yeah, so it’s a very awkward and interesting time for that to happen. Because I trace the Chinese governance like discourses very closely, like regulatory documents, like policy discourses. And they’ve they were very much coming to more of a similar degree of prioritization and understanding, more so, much more so than before, about how scary frontier AI risk can be and therefore Some organizations like Shanghai AI Lab and some social enterprises like Concordia AI were actively testing the frontier AI models, which we didn’t see happen at that rigorous or at that much of a like level. But they were doing that. And so there’s almost this momentum for the Chinese state, for the Chinese regulators to put more support behind this. And that this moment in time, with we have this very high profile resonation from Dario Amodei from Anthropic. And the subsequent conversations around human extinction, like 10% risk or how many how much percent risk. People are talking about it. And it’s a really important time to think about AI safety as a common topic of concern globally. And just leading up to the summit, you know, Anthropic CEO decided to put out an essay calling for pacing, which is fine. Although, as Trump was saying, it very much falls on their responsibility to do it, being The absolute at the absolute frontier, anthropic itself. but the fact that he kind of rehashed some of the very unhelpful framing that China and the US are completely at odds with each other, US must prioritize winning the AI race, and that China is authoritarian, therefore, like we gotta throw stones in its ways through chip export controls, as if that had worked the first time. And so I think what that prompted is that it sucks some of the goodwill. You know, the air of goodwill out of the conversation as the two governmental sides were trying to kind of stay a little bit calmer before the leaders meet. And so what happened is also that it just got- there’s more animosity and the more gossipy parts of the state media from China had to come out and respond. which doesn’t represent mainstream like policy thinking, but it kind of had to respond to him strongly. and so yeah.

Grace Shao (07:00)

Do you think there was a commercial interest for him to be doing that? It just doesn’t feel like right? It’s it

Kristy Loke (07:04)

I think so.

Grace Shao (07:06)

doesn’t align with the overall AI safety narrative that, you know, we were talking about previously.

Kristy Loke (07:11)

Yeah, I think at this point in time it’s kind of dangerous to think about safety, to mix too many things into the bucket of safety, right? Safety in its essence should essentially be making sure that these very capable models, which are s at times very often, black boxes, not very interpretable, we don’t really know what’s going on in there. As they advance, we want to make sure that they’re somewhat alignable to our values, somewhat controllable so that we know what’s happening and can stop it getting out of hand. These are very scientific pursuits, right? These are not political issues. And so I think it’s convenient for him because a lot of people working within these companies are a little bit different from how like DeepSeek or Moonshot came about in China, which is very much very much like domestic talent that got really AI-pilleded or AGI-pilled in some sense. And Anthropic and OpenAI is quite different because there is a huge community of people. Who think that AI is probably the most important existential thing to work on because it can go wrong very quickly and very, very wildly. And so that they devoted their career, their lives into making sure that it goes well. And the way that they think it can go well is if they are in charge, if they are the leaders, if they’re the powerful ones that they can make the rules. So think about it from the perspective of the government. Of course, the Trump administration’s like, I don’t love this. Like, where’s my role within this versus you? You’re trying to kind of usurp some of the power that we should have, right? And so, I think there is a little bit of a POW struggle there, as we can see. And Anthropic has you know, Dario Amodei has just a long history of not liking China. And so there’s also a bit of a struggle over there. But a lot of people have pointed out, and I agree, that the timing is kind of suspicious because the US Data center build out is running into a lot of problems, a lot of public backlash. I mean, budget-wise, it’s also, is it justifying? You know, is it like a bang for the buck? And so it’s a convenient time for them to slow a little bit down. But that doesn’t, it doesn’t mean that Jacob Coxson’s wrong. It just means that there’s a lot of incentives that are layered onto AI safety in a way that is not helpful.

Grace Shao (09:19)

Yeah. I totally agree. I just wanna bring it back to the Trump-Xi Summit. Do you think there were anything that was a bit of a surprise or shock to everyone that’s been watching this space closely? It did feel like, you know, the media coverage from both China side and the US side were quite positive. You know, there were even cute videos going around with Melania in Pengliu and you know singing along with children. The overall kind of vibe that was trying to be pro that was portrayed even by the state medias, whether it’s from the China side or even the White House side, it seems quite positive. You know, there was an attempt to reignite this so-called friendship in their speeches that they use a lot the term friendship. I think Xi Jinping said, let’s hope the friendship flower blossom or something like that, you know, at the state dinner speech. Anyway, beyond the vibes, were there any actual surprises? Were there anything that was really concrete towards AI?

Kristy Loke (10:15)

Yeah, it’s interesting that that’s your take, because from I I I agree that the photos seem pretty positive from like both sides. But I was a little bit surprised, maybe not surprised, that a lot of the main outlets in the West, you know, like a lot of the big ones, were actually trying to temper or kind of put a lid on people’s expectations around this summit. I think in a way, it kind of undermines the fact that they are still working on AI together, despite how close the competition actually is today. And in a way, you know. Yeah.

Grace Shao (10:49)

Gonna jump in. Sorry, I’m gonna interrupt. What I meant was the state media from China and the White House official videos,

Kristy Loke (10:54)

I see.

Grace Shao (10:57)

throwing this back

Kristy Loke (10:57)

Hmm.

Grace Shao (10:57)

to you is just like, was there anything concrete that came out of this that would actually impact AI?

Kristy Loke (11:01)

Right. Right. Maybe I’m just like the view that you asked me a few questions back, like how I think about coordination. And so my current project for a little bit now, because essentially I started the project with Stephen Casper at Harvey and at Kennedy School. We were initially interested in open model governance, but when we went to this international safety Exchange conference in Singapore, where a lot of like kind of big shots in research and safety across the world were all together. And we’re kind of like responding in real time to the summit, the Xi-Trump summit. And there’s generally a sense of optimism that some things can happen and that cooperation is somewhat possible. And then I look at my field and I look at people around me, and I was like, we’re extremely unprepared for this moment because a lot of people have been working on. compute governance, which is assuming that China’s going to be far behind and the US gets to lead, like how things are distributed and you know what safety would look like. And this is not reality. And so we actually need to think a little bit deeper about what actual coordination amounts peers would look like. And so we kind of like reject our project and focus more on frontier AI governance happenings within the two countries. And as we look at that, we’re like despite rhetoric, they’re actually concerned about similar things. And so going into this summit, I was actually expecting them to slowly move up the ladder of coordination. And they’ve already stepped up the first one, which is agreeing to have continuous dialogue, which is great. And then the next step can easily be doing best practice exchange. So I’m anthropic or I’m an OpenAI and this is how I do red teaming for my model. for them for the most risky kind of AI. And then China can be like, I’m deep sea, can I do this, right? So maybe through some intermediaries, they can start sharing that so that they harden defenses on both sides. And then maybe we go further up the ladder and then we can have something yeah, we can go further up the ladder and pursue something more ambitious like common standards that you were talking about. Common standards are a bit tricky because at this point in time, if the US say this is my common standard for open models, and China say this is my common standard for closed model, it’s not gonna work, right? Because they have asymmetric cost in terms of agreeing to

Grace Shao (13:19)

Right, yeah.

Kristy Loke (13:20)

something like this. So it’s a little bit tricky. But if they work their way up and actually have the experience and trust and bilateral goodwill, as we have seen in some of the photos, then we can move up the ladder quite quickly, potentially, because AI moves very quickly. And Trump’s presidency is very much an AI presidency. And C’s been AI-pilleded since twenty twelve. So I think twenty seventeen. So I think there are actually a lot of shared stakes, despite some, you know, clear competitive differences.

Grace Shao (13:51)

How has he been AI-pilled since twenty seventeen? Tell me more about that.

Kristy Loke (14:13)

Yeah, so Xi Jinping has been talking about like AI and emerging technologies in various contexts. I think the most interesting one is probably around 2017. This is strategic concept or phrase called unseen changes of a century. And that’s when a Chinese leader, you know, mostly Xi, he was talking about emerging technologies like AI and geopolitical shifts underway, perhaps not of China’s choosing, but just like how things happen to be. which is that the US seems like increasingly, for domestic or foreign reasons, is no longer the sole hegemon because China is rising very steadily. And so what he then expects is that this creates a lot of risks and opportunities. And part of the solution comes from grasping technologies that are very promising, that can then lead to a transformation in productivity, which sits at the heart of a lot of strategic. goals, right? Without a strong economy, you kind of are out of the competition. and so that’s really something that’s been at the back of his mind, the front of his mind for a long time. He’s talked about it in various occasions. and then by yeah, and then by twenty twenty, I think there’s a refinement in China’s AI strategy that I see. But yeah, he’s been AI-pilled for a long time.

Grace Shao (15:27)

and like this is a perfect segway. I want to bring it to China then you talk about the two AI visions. There’s a China version, there’s a US version. So let’s look at the China version first. You are often arguing in your work that China doesn’t necessarily see AI the same way as the US. How Do the Chinese leadership or even the country as a whole, if you had to overgeneralize, see

Kristy Loke (15:46)

Hmm.

Grace Shao (15:47)

AI? I think it was really good context that you kind of provided just now that Xi Jinping himself has seen AI and frontier technology as an opportunity to bolster the country’s, you know, status, influence, economic growth, et cetera. But beyond that, is it really just that China is more pragmatic looking at AI? How do we understand that?

Kristy Loke (16:07)

I think pragmatism is a pretty good word to sum up a lot of kind of behaviors within Chinese state when it comes to innovation. And my focus mostly is on the state, although I’m also very interested in state market relations, because you know, you gotta care about DeepSeek if you care about AI, for example. and so the reason why I find the state so interesting for the question of governance, for example, is because the state can ultimately jump in and say this is too risky. Putting an end to everything in a way that we don’t necessarily see the US government being as incentivized to do it as abruptly as decisively. So that it really is like what informed my focus. And so when I talk about the Chinese state’s AI vision, like the China’s AI vision is very much in relation to the state. And so from that front, what I find interesting is that they actually articulated this in 2020. just a little bit of background, right? So that’s like roughly a year or two after the first like export control stuff or like the the sanctions against CT and Huawei, China’s telecom companies, that kind of woke China up to core tech innovation being really important. So that’s one part of the background to this 2020 strategic concept. There’s another background, which is that in 2018, 2019, 2020. I was actually in DC around that time for my exchange, for my master exchange. And a lot of the discussion in AI was about biases, right? So after a few years of being very excited about AI, after the AlphaGo moment, leaders around the world and people who are really AI-pilleded are starting to realize this is not a simple technology. There’s like a it’s dual-edged, right? There’s a negative part to it as well. And so what happened in China is that there was a There was an incident, like a series of incidents that caused quite a bit of backlash, which is around AI platforms that were building like basically delivery apps that are not very humane, right? They’re the algorithms are not giving the drivers enough time to actually do their tasks safely. And that cost a lot of social discussions. And so that was really the backdrop. And so going into this, this the strategic concept, those were the two things that happened. And in terms of what the concept is about or what the strategy is about, there really are four prongs. So instead of fixating on AI or AGI building and fixating on racing towards the frontier, China’s model is very much we’re gonna focus on four things because four things are very important, like none of them are not, right? None of them can be dropped. And so the first one is really about getting as close as possible to the technological frontier. And this is something, this is a long-term goal for them. And in the Tsinghua explainer of this for prong strategic concept, they were really talking about this because they think that scientists in China should start touching high and aiming high as a goal that they should strive for, because that’s just better for competitiveness and technology in the long run. But it’s really interesting because the emphasis is on being close to the frontier instead of always chasing the frontier. Because China simply maybe cannot afford to do that, right? And then the second part is also interesting because it’s actually about innovation-led development. Basically asking scientists and technological practitioners to focus on the benefits of the technology. Like, are they actually translating it into productivity gains? If they’re not, then it’s kind of a problem. And they also talk about this positive loop between great technology. And productivity gains, therefore creating better technology. And so that’s very much like the thinking behind AI plus, as China’s diffusion plan, for example. The third component is also interesting and a sign of the times, which is security around supply chains for core technology, i. e., chips, but you can think of other things as well. And that also helps us explain, helps explain why China rejected some H200 orders, right? If all you’re thinking is AGI building for tomorrow because AGI is going to change everything, you don’t, you shouldn’t really take say no to these orders. You should take all the chips you can. and you should probably also give every single chip in China to DeepSeek can get them to chase the frontier. But China’s not doing that. And then the fourth one is the most interesting one and close to my heart because it has to do with governance. The fourth one is a new addition in 2020, which didn’t exist in 2018 when this concept was previously called three orientations and instead of four orientations. So the fourth prong is about protecting or ensuring people’s livelihood, life, and well-being. So essentially at the end of the day. Because China believes in general purpose technology and they think that AI is the solution to a lot of economic problems, they need people to be okay with the technology, right? So that’s one very practical component. The other component is almost historical cultural approach to looking at what technology is for. And I think this is pretty human, right? Like a lot of people have the same realization in the kind of AI conversation we see, which is that if technology is not Producing benefits to people, what’s the point of technology? Verbatim. They talk about that in that official explainer. And so that’s really interesting because it explains why China’s been doing so many vertical AI laws around AI agents, humanoids, anything you can think of that has to do with AI. And especially if we look at the AI companion law that they passed a few months ago. I thought that was interesting because the speed is insane, right? So within I think one month. major incidents happen in the US, some suicides of very young kids happened because of AI companions. And then within a month, they drafted this document saying that maybe we should ban this for a certain age or you know, help look after these kids before they get access to it. And then within another four months, I think it was finalized and passed. And it also led to quite a few big tech companies dropping out of the AI companion game. So a very interesting multipronged approach to AI development that we can see from China.

Grace Shao (22:17)

that’s super, super informative. And I know this is something you wrote extensively about in your recent op-ed for the information with Ryan Cunningham. really well written article. I urge everyone to go read it. I really resonate with the last thing you just talked about, the fourth point, which is something I’ve been writing a lot about as well. Like the technological memory and how people actually view the relation between the government and the technology is really different in China because For better or worse, when we saw the internet crackdowns or regulatories c regulatory probes, like it hurt

Kristy Loke (22:48)

Mm.

Grace Shao (22:49)

the capital market. Like no one’s denying that. And USD

Kristy Loke (22:52)

Right.

Grace Shao (22:53)

like investors freaked out and like pulled out of China. No one again is denying that. But the reality is for the average person who is just average consumer, what they saw was that, actually our rights are being protected by the government. So we can like, you know, the big tech can no longer force to choose one, which means you must pick one platform, not the other, between Alibaba

Kristy Loke (23:15)

Mm.

Grace Shao (23:15)

and Tencent for the merchants. So you protect small businesses. users then therefore can actually you know, actually get a lower price because then the platforms will gauge into a price war. So in

Kristy Loke (23:25)

Right.

Grace Shao (23:25)

the end, for the average consumer, it benefits them. The same way that the education sector gets wiped out. And I love how the media, okay, I know the media usually focuses on only the negative of an industry being wiped out, which is again true from a capital market perspective. And from a governor’s perspective, this looks scary. But for the average person, it actually means that for the people who had the means to pay for extremely expensive private education.

Kristy Loke (23:49)

Mm.

Grace Shao (23:49)

That leverage they had is taken away and the level like the playing field is leveled again for the average person and the leads. I mean, in theory, that’s what they want to do, really. And I think again, people and outside of China sometimes forget the ultimate goal of the government is really in the interest of the mass and not the elite, which is very different. I would say,

Kristy Loke (24:11)

Mm.

Grace Shao (24:12)

You know, from what we’re seeing in the West, which is why we’re seeing such a pushback and uproar toward like, you know, a pushback towards the big tech and the so-called elite and the capitalist. You know, there is a so socialist movement, even in the tech space, where they’re saying that, you

Kristy Loke (24:26)

Right.

Grace Shao (24:26)

know, like right. It’s just very nuanced and I think something you captured for sure that I’ve not seen a lot in mainstream coverage. So just wanted to add on that a little bit. But okay, let’s take a step back. Following me, Rand, let’s take a step back. I want to try to focus back on you. You talk about there’s the China AI version. What is the American AI version if we had to understand it?

Kristy Loke (24:48)

Yeah, so I actually relied on like chatting with my co-author a lot about this. And the way he boiled this down, which I thought was pretty excellent, is it’s really a concept of power, right? So Silicon Valley has a vision of power. and the typical model of development for like success in business is very much about monopolies, monopoly building. So we see it in Facebook, we see it in open AI, and as American businesses go around the world, you know, they also win a lot of money and you know, like and and you know, that is a positive loop in some sense, right, for the economy. And so that’s a vision of success that I think has worked for a long time. The problem with AI currently, if we’re, you know, kind of comparing US and China, is that because China is also very competent technologically and is able to do these open models, right? Not because the state says you gotta do it, but because the state actually nurtures the open model ecosystem and also, because there are a lot of talented engineers who know what they’re doing in China. And so we see just cutthroat competition there. And so the price-making power is no longer in the hands of like these monopolies, for example. And so it kind of challenges that model somewhere. And then another element I think it’s important is that AI is different. If we think about AI not as a weapon or whatever, it’s very much not a normal technology beyond that. Right. It’s not a normal technology in the sense that it needs to be used in the world and it needs really to create value through the economy, through people and businesses. and a lot of people feel kind of left behind already in the you know previous wave of development, like with the social media companies and all that. And so there’s quite a lot of unresolved negative sentiments around technology and around a trust deficit with Silicon Valley and with the government. That hasn’t been resolved. And so when you push another technology onto people, I know this because my husband’s told to use a lot of AI and it was in his work. And I think a lot of other companies are doing the same. or people are having the backyards with data centers built upon it, it just creates even more bad blood. And so if I can talk about the like if I if I can summarize the US AI vision, I think it’s very much about leading, right? Leading by being by building the most advanced AI systems. and this is good for business in Silicon Valley, this is good for American hegemony. but as Ryan was writing in that piece, the problem is that the language of power doesn’t speak to those who are already disempowered, right? So most people do not feel like they’re benefiting as much from AI and therefore are pushing back on it.

Grace Shao (27:24)

That’s really interesting. It’s not just the tangible benefits that people cannot feel, but it’s actually the intangible that people are also disassociated or disconnected with as well. tell us about actually how you got into all this. I just while we’re having this conversation, I just found it quite fascinating. Like tell us a bit about yourself, tell us about your background and how you got into AI governance.

Kristy Loke (27:43)

Yeah, so I grew up in Hong Kong and I thought that US-China relations is probably the most fun, nerdy thing to study. and around when I was doing my undergrad in the UK, the news just wouldn’t stop talking about US-China AI arms race. And so I was like, there must be a US-China AI arms race. I want to study that facet of US-China relations. and in the subsequent years I realized, you know, this might not be the best label to what is actually happening. But anyways, that got me interested. And I did my master at HKU. and I was very much trying to understand how US-China relations and competition gets kind of changed by the arrival of AI or the growing importance of AI. and then since then spent some time at various AI governance orcs. Including at GovAI, in Oxford, now in London. and when I spent time there in early 2023, everyone around me was AI safety pilled. So everyone was trying to do their best to make sure that we are putting out the governance measures so that a safer advanced AI, when it arrives, you know, like we’re we’re sorted, we’re fine. And one key research question within that basket is actually The race to the bottom thesis. The race to the bottom thesis is that, you know, if you’re a great power, you probably want AI for strategic reasons. And as you want AI, you probably will throw all the resources added to make sure you win. So very much the AGI race kind of framing. And initially I really bought into this as well. And so I started my project 2023, looking at China’s responses, again, the Chinese state’s response to AGI as a narrative and ChatGPT and all that. Just the longer I look at it, the more I realize if you throw a power centric framework at the Chinese state, somehow it doesn’t work. Because they think in a very long-term systemic sense and they also very problem focus. So if there’s a problem in terms of the economy and in terms of where they want China to be and and you know, like the the the vision of success, if I can characterize it for China’s very much How do they transition the economy so that you know it becomes more sustainable and then everyone has food to eat and therefore it’s a more stable country and stronger and all that? And so that power theory just breaks down. And so I’ve been very inspired since to try to build a better framework. And I happen upon the four orientations and I’ve been preaching it to everyone that I can. I can talk to that will listen to me and say, you know, China’s been behaving accordingly for the last three years when AI’s changed a lot. and become even more important. So maybe we should use a different, more empirical frame to understanding what China wants going into the future.

Grace Shao (30:22)

And how the AI arms race maybe is more of a one-sided reflection, a projection of how the America sees this versus how the China actually sees. Because I get a lot asked a lot whether I ever hear that framing in China, and I frankly just don’t. There’s never a

Kristy Loke (30:36)

No.

Grace Shao (30:37)

researcher or business who says, I just wanna win America. I’ve never heard that, but whenever I’m out in the US, that’s often the first thing I hear. so it’s really interesting. It’s a projection of the their own culture or own ideology. But also, I don’t know if this is contentious, but I think it’s also because of which we’ll not go into, but because of different political systems, there’s less of a PR need is also. I think the Chinese

Kristy Loke (30:59)

Right.

Grace Shao (30:59)

government never really needs to project or try to do PR for their own initiatives because they just do what they want to do. and you know, versus, you know, a lot of times within the US, different parties have to preach different rhetorics for different If you must clientele, you know, so

Kristy Loke (31:18)

Maybe.

Grace Shao (31:19)

so essentially like you you have a bit of a PR drilled into this. So the arms race is a very easy narrative to kind of push out, right, to the stakeholders. I’m using a corporate stakeholder kind of lingo, but do you know what I mean, right?

Kristy Loke (31:31)

I think what’s really interesting to me is like there’s a poll and I s shared on on X a few months back, which is that Americans, I think throughout across the different ages, age groups, actually feels I might be wrong across the age groups, but like just in general, majority of them seem to think that China’s leading in AI. And then if you look at other polls

Grace Shao (31:51)

That’s so o odd and interesting.

Kristy Loke (31:54)

right? And then if you look at other polls, they are saying that, like a recent poll, Americans, especially the younger ones, prefer the US and China to get along, right? and so that cult war, Cold War mentality, or that you know, like if it is zero sum mentality might also be an age thing. On one side, I think on a level, I understand it because you feel like you’re strong and you feel like you’re the hegemon. It’s a hard place to kind of give up power and admit that you’re sharing power with someone now. Which is why I think the pageantry around the mutual respect between Xi and Trump

Grace Shao (32:28)

Mm mm.

Kristy Loke (32:29)

and all these state media stuff that we’re seeing is very interesting because Trump just gets it. He just understands what Xi needs in terms of mutual respect. but yeah, I think there are many factors, yeah. Yeah.

Grace Shao (32:38)

And I think equally I’d say the same, right? She’s giving him all of the mianzi as well. So yeah. Okay, so but back to AI, AI governance. How

Kristy Loke (32:48)

Mm-hmm.

Grace Shao (32:48)

Do we understand the risks and the prospects of global cooperation and engagement around AI since this is now often news and you know, what should people outside of AI governance actually care about?

Kristy Loke (32:59)

How should we think about? Yeah, so again, as I was saying, a lot of folks, you know, I know some of these people, a lot of folks have been caring about AI going well for a long time and they kind of make their life around this goal, right? I know people who really prioritize, you know, maybe the way they save money and the way they pick partners so that they can focus on making sure that things go well. Which is great. I’m glad that people are working, you know, to help everything go smoothly and to lower the risk. But at the same time, you know, when scientists or researchers at Anthropic coming out and saying there’s X-risk, it’s gonna kill people, you’re really not giving the normal public the space to understand what is actually happening. So on one hand, I think it’s important to have everyone understand the stake. As this kind of like Chinese cyberspace administration leader was saying recently, he listed out five key risks, and the first risk from AI was for him interpretability, because AI, because we’re using deep learning type of AI now with all these generative AI models, we can’t really understand what’s going on inside, right? It’s a black box. And so that creates a lot of control and alignment problem. So I think on that level, definitely. Definitely a problem, definitely something that people go to work on. What is also a little bit riskier or a lot riskier is some of these laps. because for a while, if you scale the model, if you scale the if you scale the data, you’re yielding a lot of great capability benefits. But at this point in time, that is not working as much. And so they’re trying something new, which is recursive self-improvement, getting these relatively smart models to train their own next smart models, right? And that’s when we can. Potentially really just lose track of what’s happening. Right. If they’re doing this, why would you be incentivized to try to have our slower brain follow how they’re doing that? Right. It just kind of doesn’t make sense. And so RSI is of concern because of that. And most leading labs are exploring with that. And so I think we should be concerned. We should get our governments to hold them accountable, hold these companies accountable. but Also at the same time, I urge AI safety people to think a little bit more when they use the word existential risk or hum humanity is gonna ext go extinct or whatever. they kind of gotta care a little bit more about the audience. and so if you wanna get the policymakers to understand a stake, great. But if you want to educate the public, you probably want to go a little bit easy, if that makes sense.

Grace Shao (35:34)

Kristy is there anything else you think we’re missing that we haven’t discussed that’s very important for us to understand AI safety and AI governance at this point?

Kristy Loke (35:40)

Mm. I if we have time, I think it’ll be fun to chat about how AI governance works in China, because a lot of the myth or the framing around

Grace Shao (35:49)

Mm.

Kristy Loke (35:50)

governance versus development in the US is that they’re

Grace Shao (35:52)

Mm-mm.

Kristy Loke (35:53)

like they’re like incompatible, right? And what

Grace Shao (35:56)

Right.

Kristy Loke (35:57)

is interesting in my observation about China is that from the get go, from twenty twenty three onwards, when they need to decide on what to focus on, they decided that the two are actually one and the same. Which is like very, a very I don’t know if it’s a very Chinese way of going about it, but it’s it’s very interesting because it’s like a philosophical difference in terms of the way that I see regulators talk about AI risks and talk about AI governance, the way that I see C talking about using the study session on AI post-deep seek to talk about both development and governance was striking. And the way that I kind of reasoned through it is the Chinese companies and the Chinese state are trying to move fast. without breaking things. And how do you do that? Well, you have a lot of iterative laws. You include the experts for all your law drafting, and China’s doing a lot of that. Just as an example, the AI safety governance framework from a main national standards body involved over 20 civil, like as in private sector and academic and research organizations to help them with their drafting. And then there’s also a third component that is really interesting, which is that China looks to the West and look to the world when it comes to the frontier AI governance discussions. So if you’re talking about pacing, if you’re talking about X-risk, they’re taking notes, right? Which is a good way to kind of stay on top of what’s happening. And at some point in time, as China gets closer to the frontier, I can start more actively contribute and maybe maybe lead some of the conversations. But like I think the contribution is gonna go up. In terms of AI safety research coming out of China. The last component to this, which is maybe of interest to your audience, is how do you do all of this and still move fast or like allowing innovation to go fast, especially when you operate in a system where you care a lot about content security. You don’t want people to say bad things against the government. The way that they go about this is simple. It’s very simple. It’s essentially drawing the red lines really early, very clearly and very narrowly. So in 2023, they made it clear that content security still matters when it comes to generative AI. And that’s a red line, but everything else, you know, as long as you show goodwill that you’re trying to do good governance, then you should be fine, unless if it’s a repeated offense. So that’s really interesting. And I think within this part, which is like narrow red line drawing. I find also a potential solution to front AI risk governance, because if China can again keep those red lines light, maybe only for the most capable models, maybe only for the companies that are attempting the riskiest ways of developing AI, do they get those like kind of like harder oversight? then they can implement that probably faster, right? So yeah, that’s how it works in theory or in practice.

Grace Shao (38:49)

That’s very, very insightful. Thank you, Kristy. I wanna end on one last question, which is a question I ask every single guest that comes on. What is one difference to view you hold as something you think is a bit non-consensus?

Kristy Loke (38:59)

I think it relates to some of what we’ve talked about, but I saw a headline, I think it’s from the FT saying that Trump and C are maybe the worst people to be leading this part of the AI problems that we’re seeing, AI governance needs that we’re seeing. And I actually disagree. I think because Trump is such an ultra pragmatist and he is very much the AI president where His, you know, people who have backed his presidency are very much invested in AI going well. And so he actually has a strong stake in implementing some level of governance that helps with security. and when we look at C, for example, I think he is very much also similar in the way that he has been AI-pilleded from the get go and he really sees China’s future in climbing up this value, like higher value production and industrial upgrading as the only path forward. And so they are very much aligned in terms of a lot of things when it comes to frontier AI governance. And I think the other differentiated view is that coordination is possible and maybe within reach. but at the same time the risks are growing. So That’s the good and the bad.

Grace Shao (40:11)

So you sit in Toronto. I was actually just wondering, what is your view on how these middle powers should balance AI governance and AI growth? you know, how should they balance the relationship between China and US?

Kristy Loke (40:24)

Yeah, I just did a podcast earlier talking a little bit about this as well. I think it’s important to on the developmental side, it’s important to remember that you two have the ability to make the change you want to see. And in some ways, you know, for Europe’s case, for example, maybe they need quite a bit of reform to make sure that innovation actually goes faster. Companies that need a little bit more help get the help they need. and so you probably need reform to get there so they can apply AI and get the diffusion benefits without building all these frontier models necessary necessarily. and then on the coordination side, we are very much in the big players game at this moment in time because AI is extremely unpredictable in terms of what’s going to happen next in terms of risks. And so we need the frontier players to get at the table and talking with each other first. And in terms of what middle power can do or should not do, I think some of the example actually comes from maybe Singapore or UK AI Safety Institute. It shows that you can win a lot of soft power and real benefits to kind of sitting at the heart of global governance, but that only gives you so much power because the US is currently considering rolling back some of that transparency that UK AI Safety Institute currently gets. So Bit of a mixed bag.

Grace Shao (41:41)

Yeah, that’s really interesting. I have actually written quite extensively on how Singapore has played a very for lack of better word, pragmatic strategy and courting both sides as well and how they’ve actually come up on top, like and built like, you know, their own ecosystem. They not only host all the OpenAInthropics of the world’s APEC headquarters in Singapore, but they are also now building their sovereign AI on the Chinese open source stack. So it’s very interesting. They’re not rejecting any talent, they’re not rejecting any capital. They’re

Kristy Loke (42:10)

Mm.

Grace Shao (42:11)

open for opportunities, but they’re also very mindful of building something that’s really aligned with their own values and safety standards for their own country versus purely depending on one stack versus the other. Very interesting. Okay, Kristy,

Kristy Loke (42:24)

Hmm.

Grace Shao (42:26)

Thank you so much for your time today. I learned so much and I was very, very insightful and timely. I’m gonna try to push this out as soon as possible.

Kristy Loke (42:33)

Sounds great. This is so much fun. Thanks for having me.

AI Proem is a reader-supported publication. To receive new posts and support my work, consider becoming a free or paid subscriber.

Get full access to AI Proem at aiproem.substack.com/subscribe