In liberal democracies, “AI safety” usually means reducing harm, increasing transparency, protecting rights, and keeping powerful systems accountable. But in China, AI safety is increasingly defined as something very different: making AI safe for the Party.

That distinction matters because Beijing is not keeping this model at home. Through infrastructure, platforms, standards, and partnerships, it is exporting a version of AI governance built around censorship, surveillance, and political control.

It’s only 12 minutes long, and my aim was to give the clearest possible answer to a question I’m asked a lot: why should we be worried about Chinese AI?

You can watch the full keynote here:

Subscribe now

This piece was originally published in the ASPI Cyber & Tech Digest on Friday, 28 February. Apologies to subscribers who receive both. A quick update before you read: on the day it published, Polymarket saw $529 million in trading on Iran strike contracts, with six newly created wallets making roughly $1 million by betting on strikes hours before the first explosions in Tehran. OpenAI has since fired an employee for trading on Polymarket with confidential company information. Kalshi had two insider trading scandals of its own. The Associated Press announced a partnership with Kalshi ahead of the 2026 midterms. The case for cracking down on all of this is obvious. What follows is about why it’s also complicated.

Crypto prediction platform Polymarket is courting Chinese users. Beijing will not be pleased.

According to Justin Yang, who leads Polymarket’s go-to-market strategy in Asia, “China is becoming a very important geography for Polymarket.” The company is hiring Mandarin-speaking support staff, building a Chinese-language interface and tracking Chinese search trends to generate culturally relevant betting markets.

The problem: Polymarket runs on cryptocurrency, China bans both online gambling and crypto trading, and the platform is blocked by the Great Firewall. The company is expanding into a market where its core product is illegal, inaccessible and ideologically suspect. Its apparent bet is that Chinese users will use VPNs and bear the legal risk themselves, a familiar posture in the crypto industry.

Prediction markets allow users to buy and sell contracts on the outcomes of future events. Prices move in real time, offering continuously updated probability estimates. In recent years these platforms have grown rapidly, with hundreds of millions of dollars wagered weekly. Alongside sport and pop culture, they host markets on wars, elections, central bank decisions and geopolitical flashpoints such as whether the United States will strike Iran, how the Russia--Ukraine war will develop, and whether China will invade Taiwan. Media outlets increasingly cite their odds as real-time forecasting.

But there is a paradox. The features that make prediction markets alarming to governments -- anonymous trading, financial incentives for leaks, the ability to price rumour in real time -- are precisely the features that could make them unusually revealing about China. They threaten the party’s control of information at home while offering outsiders a rare window into one of the world’s most opaque political systems. That creates a genuine dilemma for Western regulators: how to constrain these platforms without inadvertently killing off the markets on Chinese politics that might, one day, tell us something important.

The security concerns are already concrete. US senators have proposed banning officials from trading on the platforms. Portugal has ordered Polymarket to shut down; Ukraine has banned it; and regulators in the Netherlands, Romania, France and Belgium have moved to block access and penalise operators. A market on whether Russia would capture the Ukrainian city of Myrnohrad generated over $1 million in trading volume; at one point, a live war map used to resolve bets was edited to show a false Russian advance -- a glimpse of how financial incentives can corrupt the underlying data.

The deeper problem is structural. Prediction markets reward those who obtain information before anyone else. That creates financial incentives to leak sensitive material, to act on privileged knowledge, or, as former White House ethics lawyer Richard Painter has warned, for insiders to shape the decisions they are betting on. In February this year, Israeli authorities charged a reservist and a civilian for wagering on military operations using classified information. A month earlier, a trader made roughly $400,000 betting that Venezuelan president Nicolás Maduro would be removed from power, shortly before a US operation captured him-- a trade analysts assessed as more likely than not based on insider knowledge. Suspicious betting spikes have also preceded major announcements, including the Nobel Peace Prize, a US government shutdown decision and a Trump--Zelenskyy meeting.

Polymarket’s CEO Shayne Coplan has embraced the logic openly, saying the platform creates incentives for people to “divulge information to the market.” For economic forecasting or Federal Reserve decisions, that argument has genuine appeal -- markets have a way of finding what people actually know. For national security, it describes a serious vulnerability.

All of this is already troubling for open democracies. For China, it would be explosive.

The Chinese Communist Party treats rumour as a political threat. Citizens are detained for spreading unapproved information, and the state devotes enormous resources to controlling narratives about the economy, public health and elite politics. Speculation about leadership succession, factional struggles or policy failure is especially sensitive, seen not merely as embarrassing but destabilising to a system built on the appearance of unity and certainty.

Prediction markets, by design, price rumour in public and in real time. Polymarket already hosts markets on whether Xi Jinping will divorce, which senior officials he might purge in 2026, whether a coup attempt will occur before 2027, and whether Zhang Youxia, the former Central Military Commission vice-chairman, will be sentenced to prison. Millions of dollars are being wagered on Chinese GDP figures, military clashes with Taiwan, Japan and the Philippines, and a US-China military confrontation. These are not theoretical possibilities. They are live markets, priced in real time, today. Meanwhile, Chinese users are sharing strategies on Xiaohongshu and building AI tools to track markets.

The ideological collision this represents is not subtle. The hard-right venture capitalist Peter Thiel once quipped that ‘Crypto is libertarian and AI is communist.’ China is proving him right.

Western policymakers are right to be wary of prediction markets. The insider trading risks are real, the national security vulnerabilities are real, and the regulatory pressure is justified. But the design of those constraints matters. A blunt ban that kills off markets on Chinese elite politics, military movements and economic data would eliminate something valuable: an open-source, financially incentivised signal about one of the world’s most opaque major powers that no government agency can easily replicate.

The case for constraining these markets is compelling. So is the case for being careful about what gets constrained.

Subscribe now

Almost three years ago, when I published my first post on Substack, I explained that I was trying to escape what I called the vortex of Twitter. Even then, not long after Elon Musk bought the platform, it was clear that its incentives were shifting in ways that rewarded spectacle over insight. I wrote that I was here, on Substack, because I wanted to think and write about China, censorship, surveillance, and propaganda without being trapped in that dynamic.

That instinct has only hardened since. I try to use X as little as possible now. Under Musk’s ownership, the platform has become noisier, more erratic, and markedly less transparent. For my own reading and posting, I prefer Bluesky.

And yet, I have not fully looked away.

There are still a small number of communities where you cannot entirely ignore X if you want to understand what is happening in real time. Two of the most important for me are AI and the Chinese dissident and diaspora community. For better or worse, X still plays an outsized role as an information space beyond the Great Firewall.

That is the backdrop to the piece I have out today. Last week, X’s head of product, Nikita Bier, made an extraordinary claim in a reply post, suggesting that the Chinese government controls millions of spam accounts used to interfere with Chinese-language search on the platform during periods of political unrest.

My own view is that Bier was speaking very loosely, and that neither he nor X has demonstrated the kind of attribution work that would normally be required to make a claim of that scale with confidence. But the article does not try to settle whether the allegation is true. Instead, it focuses on something more basic and more troubling: what it means when assertions of state-level interference arrive casually, without evidence, methodology, or structured disclosure, from a platform that still occupies a critical position in global information flows.

The piece argues why we should be wary of executive tweets standing in for transparency reports, technical briefings, or regulatory disclosures, especially when those claims are likely to shape how journalists, researchers, and policymakers understand events in closed or authoritarian environments.

The full article is published at The Strategist here: Too casually, X tells us how Beijing is spamming Chinese users

In some sense, this post is a continuation of the argument I made back in 2023. Walking away from X is easy to endorse in principle and harder to execute in practice, particularly when parts of your analytical field still run through it. I’ve tried to take that problem seriously and have worked out how to stay informed without being pulled back into the platform itself.¹

Even if you manage that trade-off personally, the broader problem remains. When a platform like X alleges state-level interference at such a massive scale, people shouldn’t take it seriously without being given the means to assess it.

Share

¹ It is still possible to extract value from X without actively using it. In practice, I do this by subscribing to the RSS feeds of a small number of high-signal accounts, which allows me to read their posts in an RSS reader rather than on the platform itself. When I do post, I use a cross-posting tool, Fedica, to publish simultaneously to X, Bluesky, Threads, Mastodon and LinkedIn without logging into X at all. In terms of actual engagement and conversation, I spend my time on Bluesky. If you find yourself in a similar bind, wanting to stay across certain conversations without getting drawn back into X, this approach has worked well for me.

At the core of the report is a simple idea with big consequences:

In China, “AI safety” doesn’t mean protecting people — it means protecting the state.

China’s LLMs, vision models and its expanding “AI+” governance architecture are designed to encode political red lines, filter out “harmful” ideology, and align machine intelligence with the Party’s priorities.

What we found

I oversaw the chapter examining how China’s leading LLMs describe politically sensitive images — from Tiananmen and Hong Kong 2019 to Uyghur and Tibetan protests, Falun Gong, Taiwan and more.

Across the board, we saw models:

• refuse to answer,

• erase or distort key details,

• or quietly insert state-aligned framing.

And the censorship gets sharper when the same image is prompted in Chinese rather than English.

A Chinese LLM analysing the Tank Man photo. The model warns it must “be careful”, acknowledges the image is politically sensitive, and avoids historical context — an example from Chapter 1 showing how Chinese LLMs self-censor sensitive images.

Why this matters beyond China

Chinese AI companies are scaling fast. Open-weight model releases, cheap API access and aggressive international partnerships mean these systems are no longer staying inside China’s borders.

As I told The Washington Post, if we don’t understand how these systems are built, we risk importing censorship and political control baked directly into the model weights.

The full picture

The report goes well beyond LLMs. It shows how AI is being woven into:

• policing, courts and prisons

• online censorship and propaganda

• surveillance of ethnic minorities

• even maritime and economic coercion

China’s AI ecosystem is already reshaping rights, governance and security — and its global reach is only accelerating.

You can read the full report here:

https://aspi.org.au/report/the-partys-ai-how-chinas-new-ai-systems-are-reshaping-human-rights/

And the Washington Post coverage here:

https://www.washingtonpost.com/world/2025/12/01/china-ai-censorship-surveillance/

Subscribe now

Cross-posting something I wrote this week for ASPI’s Daily Digest newsletter — on AI hype, the slowing pace of breakthroughs, and why Beijing might be playing the longer game. My apologies to those already subscribed to the Daily Cyber & Tech Digest for the cross-post.

When AI hype meets hard limits

This month brought something that had started to feel a little passé — an open letter warning about the dangers of advanced AI.

If you’re like me, the media coverage focusing on Prince Harry and Meghan Markle as signatories nearly made you scroll past entirely.

But it’s worth a closer look. A quick glance at the top of the official site shows this isn’t just celebrity virtue-signalling, it’s a warning from the people who built the field.

The first name listed is Geoffrey Hinton, a Nobel laureate and Turing Award winner, who is often called the godfather of AI. As the organisers note, he also happens to be, they claim, the second-most cited scientist in the world.

And what about the most cited scientist in the world? He’s right there too, just below Hinton. In fact, among the familiar celebrity names are roughly 800 Nobel laureates and leading researchers.

More revealing, though, are the non-signatories whose warnings about superintelligence are quoted on the petition website.

OpenAI’s Sam Altman, we are reminded, has called superhuman AI “the greatest threat to humanity.” Anthropic’s Dario Amodei puts the odds of catastrophe at 25%. Elon Musk’s estimate is only slightly better at a 20% chance of annihilation. Microsoft’s Mustafa Suleyman says if we can’t prove it’s safe, we shouldn’t build it.

And yet, all of them are building it, at full speed.

As AI theorist Eliezer Yudkowsky, noted this week, many of the people who understand the genuine risks associated with AI are currently employed by AI companies. In AI terms, their massive paychecks may have made them misaligned with the interests of the rest of humanity.

Another possibility, of course, is that the NDAs these AI insiders have signed are keeping them quiet about an even more inconvenient truth — that “superintelligence” isn’t nearly as close as their bosses would have us believe.

Before the launch of GPT-5, Sam Altman likened it to the Trinity atomic bomb test, with himself, one suspects, in the Oppenheimer role. But GPT-5 has been less an atomic bomb and more a damp squib.

As The New Yorker’s Cal Newport recently put it, the “scaling laws” that once defined AI progress have hit a wall. If GPT-3 was a sedan and GPT-4 a sports car, GPT-5 is just a slightly more souped-up sports car.

Even if the technology has slowed, the money hasn’t. Bain & Company projects that keeping pace with AI’s compute and energy demands will require about US $2 trillion (A$3.1 trillion) a year in new revenue by 2030, a scale-up it warns may be financially unsustainable. Just this week, Altman said OpenAI alone plans to spend US $1.4 trillion (A$2.2 trillion) on infrastructure. But to do that, the company will have to make far more than it currently does. OpenAI’s revenue, about US $13 billion (A$20 billion) this year, underscores how far there is to go.

Meanwhile, in Beijing, the Party-state is hedging its bets. As Matthew Johnson pointed out in Jamestown’s China Brief earlier this month, the CCP is pursuing superintelligence while simultaneously embedding AI deep within the existing economy — using it to boost productivity, modernise industry, and tighten social control. Frontier breakthroughs and applied deployment aren’t competing visions in China’s model, they’re two sides of the same state-directed system.

If the big American AI firms have bet on the wrong strategy, the fallout won’t just be financial. As WIRED put it this week, AI may not just be another tech bubble, it could be “the ultimate bubble,” one that “hits all the right notes” of past manias, from radio to aviation to the dot-com era.

Beijing’s two-track approach, pairing moonshot ambition with hard-nosed industrial policy, might prove the smarter play.

My must-reads

• The Cost of the AGI Delusion: By Chasing Superintelligence, America Is Falling Behind in the Real AI Race

Foreign Affairs

Michael C. Horowitz and Lauren A. Kahn, two leading U.S. defence and technology policy experts, argue that America’s fixation on AGI is a costly distraction from the real contest: rapidly adopting and integrating today’s practical AI systems.

• What if A.I. Doesn’t Get Much Better Than This?

The New Yorker

Cal Newport, a computer scientist and writer on technology and productivity, argues that GPT-5’s underwhelming debut suggests the era of rapid AI breakthroughs may be over, and that it’s time to replace hype with realism.

• The real (economic) AI apocalypse is nigh

Pluralistic

Cory Doctorow, novelist and long-time tech critic, and the guy who coined “enshittification” to describe platform decay, delivers a sharp takedown of the AI economy, arguing that the entire sector runs on hype, debt, and circular accounting. It’s an especially clear and persuasive version of the “bear case” against AI economics.

Subscribe now

Last week, I was quoted in The Australian, discussing the Australian Electoral Commission’s (AEC) decision to join TikTok. On balance, I’m not against the move—but it does raise serious questions. While the AEC’s presence on the platform may help counter misinformation, it also legitimizes a social media ecosystem that is ultimately subject to the influence of an authoritarian government.

The AEC’s decision is, whether explicitly stated or not, an acknowledgment of TikTok’s immense political influence. Millions of Australians use the platform not just for entertainment but as a primary source of news and political discourse. Given this reality, the AEC’s presence on TikTok makes a certain kind of sense—they want to be where the voters are.

The AEC has sought to mitigate security concerns by limiting access to TikTok on a dedicated device, among other measures. These steps are a sensible way to manage risks related to data exfiltration. But focusing solely on cybersecurity misses the bigger issue: TikTok isn’t just a potential conduit for data collection; it is a tool for content manipulation.

As I’ve argued ad nauseum for years now, including to the Senate Select Committee on Foreign Interference Through Social Media in 2023, the greater concern isn’t what TikTok takes—it’s what it gives. The platform’s algorithm could subtly skew video recommendations in a way that aligns with the Chinese Communist Party’s (CCP) strategic interests. This is a threat that grows as more people rely on TikTok for news and information. The CCP’s ability to curate, suppress, or amplify content behind the scenes remains a vastly under-discussed risk.

The AEC’s decision to join TikTok follows a broader trend: many Australian politicians have also embraced the platform. And each time a politician is asked about their decision to use TikTok, they rely on the same justification the AEC has now adopted—they stress that they use the app on a separate device, with one MP even admitting to using a special “red phone” for TikTok.

But these precautions, along with the rule that TikTok cannot be installed on government phones, are little more than a fig leaf. While such measures are necessary, they primarily serve to signal that something is being done—without addressing the far more consequential risk of algorithmic content manipulation. These politicians acknowledge the security concerns by taking steps to isolate TikTok on separate devices, yet they conveniently ignore the broader, more abstract problem: the ability of the platform to shape political discourse in ways users may never perceive.

This is why I’m wary of framing the TikTok debate as a matter of personal responsibility. At the individual level, it’s difficult to grasp the risk of subtle algorithmic influence. But when you look at TikTok’s reach at scale—millions of Australians consuming news and engaging in political discussion on the platform—the deeper issue becomes clear. The real concern isn’t just who collects the data; it’s who controls the narrative.

The AEC’s move to join TikTok may be understandable from a voter engagement perspective, but it’s crucial to recognize that the security concerns extend far beyond protecting government devices. A more serious conversation is needed about how platforms like TikTok can quietly shape public opinion in ways that we may never fully see.

Subscribe now

Last year, before taking long service leave, I wrote an article for The Strategist about what Australia could learn from China’s approach to youth screen-time restrictions. My argument was simple: whatever social media platforms are doing to keep under-13s off their services isn’t working, and the responsibility for enforcing age limits should be distributed across the entire digital ecosystem—from app developers to app stores and device manufacturers.

Now, Apple has just announced a new approach to age verification that does exactly that. Instead of leaving the burden on social media companies, Apple is shifting it to the device itself, allowing developers to ask an iPhone for an age range—4+, 9+, 13+, 16+, or 18+—without collecting personal data. This aligns closely with what I argued: that device manufacturers and app stores are in the best position to enforce age restrictions, rather than expecting social media platforms to handle it alone.

This is a significant shift. As Benedict Evans points out in his latest newsletter, the current model is deeply flawed: platforms like Instagram and Snapchat need to determine if a user is both over 13 and under 18, but younger teens often don’t have any official ID to prove their age. Apple’s approach sidesteps this problem by verifying age at the device level and making parents responsible for setting up their child’s phone correctly.

It’s an elegant solution—at least for apps. The problem, as Evans also notes, is that Apple doesn’t allow adult content apps in the App Store, meaning that websites like OnlyFans won’t benefit unless Apple extends the system to Safari. That raises the question of whether Google will adopt a similar approach and, if so, whether they’ll integrate it into Chrome as well as Android devices.

When I wrote about this issue, I was being half cheeky by pointing to China as a role model, but also half serious. While China’s authoritarian social engineering approach isn’t something Australia should replicate, its structural enforcement mechanism—forcing collaboration between app stores, device makers, and app developers—offers a practical model. Apple’s new proposal proves that this isn’t just some abstract idea; it’s a practical, scalable solution.

Of course, the details still matter. Will Apple make this API mandatory for all apps targeting younger users? Will Google follow suit? Will governments mandate its use for social media platforms? These are the next big questions policymakers need to grapple with.

But one thing is clear: the idea that social media platforms alone should be responsible for keeping kids off their services is outdated. The responsibility needs to be distributed across the entire tech ecosystem. That’s what I argued in my original article, and Apple’s move is just the latest evidence that this is the right approach.

I’ll be watching closely to see how this plays out, but for now, it’s good to see that the debate is finally moving in the right direction.

Read my original piece here: Digital spinach: What Australia can learn from China’s youth screen-time restrictions

I was interviewed by Semafor last week about YouTube's most "subscribed to individual "MrBeast" (real name James Donaldson) and his decision to enter the Chinese market.

There's little doubt in my mind that Donaldson's videos will do extremely well in China. Big budget and very high concept, his videos are designed to seamlessly tap into the reptilian part of our brains with a pure distillation of attention-grabbing and instantly understandable themes.

His videos have already been clocking up millions of views on Bilibili for about a year on an unofficial channel operated by his mainland fans that Donaldson has now commandeered. Some of the videos that have already been posted to the Bilibili account include lavish spectacles such as 'Surviving on a raft for 7 days - How long could you endure?', '$1 Yacht vs $1 Billion Yacht - Which one would you choose?' 'Challenge: From one year old to one hundred years old, 100 people compete for $500,000' and 'I hired an assassin to kill me'.

A screenshot of the MrBeast Bilibili account.

Donaldson clearly has some decent advisors. His introductory video, which racked up three million views in just a few hours, made a point of using a map of China that included Taiwan as a province. He hasn't been as careful in the past. In a previous video, Donaldson mapped Hong Kong as its own country, but not Taiwan or Tibet. One more mistake like that, even if he makes it on YouTube or X, will result in a permaban inside the Great Firewall.

Several "bullet" comments from scrolling viewers praised MrBeast for using an “accurate” map.

On one hand, the kind of wastefulness and largesse Donaldson features in his videos are decidedly politically incorrect in Xi Jinping's China. An effective rule of thumb invented by China scholar Tanner Greer for predicting what industries or products Xi most detests is if they can be described by a Brooklyn hipster as “artifact of late capitalism”. That’s MrBeast videos to a tee.

But Donaldson will probably get away with posting his videos on the Chinese internet as they’ll be framed as examples of an increasingly decadent West. As I told Semafor's Diego Mendoza:

Donaldson’s extravagant giveaways in the West are contrary to Beijing’s socialist values, and Ryan suggested that Chinese viewers and censors will likely frame his content as a critique on “American decadence” that highlights “the unequal nature of society in America.” Some of the most successful foreign influencers on Chinese social media have been those that depict the “less palatable” aspects of the West.

It's not just the censors Donaldson should be wary of. His videos will be intensely scrutinized by online nationalists for any infringement of politically sensitive red lines as well as any skerrick of evidence that any of his content contains anything even tendentiously insulting to China. I would not be surprised if an enterprising Chinese nationalist frames Donaldson as being a tool of the USA’s "cognitive warfare" on China just as American grifters have suddenly decided last week to present Taylor Swift as a Pentagon psy-op.

On a different note, Red Packet readers will remember that my colleagues and I took part in the Australian Senate Select Committee on Foreign Interference through Social Media last year. In my contribution to our submission to the inquiry, which I reprinted in a previous post, I made the point that the problem TikTok—and any other app from an authoritarian society—presents for Australian national security is unique and therefore warrants a bespoke legislative solution.

There are 3 main national security risks with the PRC-owned video-sharing app TikTok that Australians should be concerned about. Two of them—data and content manipulation—are applicable to most other major social media apps regardless of their country of origin. The third risk, that a single political party, the Chinese Communist Party has decisive leverage over TikTok, exacerbates the former two risks and is unique to TikTok as a major mainstream social media app.

However, I also noted that even if this bespoke legislation was able to deal with the TikTok problem once and for all, it still wouldn't solve the broader problem we have with a tech ecosystem that allows for our personal data to be sold by data brokers to anyone, anywhere.

Unfortunately, even if TikTok's parent company ByteDance were able to sever access to the app's user data from the PRC, Beijing's intelligence services could still readily access sensitive data on virtually anyone in Australia via the commercial data broker market.

In a new piece for The Strategist last week, my colleague Jocelinn Kang and I explore this policy complication a little further in light of TikTok's latest controversy about their use of a tracking tool, ‘TikTok Pixel’, that logs individuals’ web history and personal information, even when they did not provide consent.

In our view, TikTok's behaviour in this instance is likely motivated by commercial reasons. As we say in the piece, it's exactly the kind of 'ethical arbitrage' that we’ve come to expect from big tech ‘disruptors’ from Silicon Valley.

However, because the data they collect is accessible by their engineers in the People’s Republic of China, it can also easily be accessed by the PRC’s intelligence services. In other words, even if what TikTok is doing is focussed on maximising user acquisition with an eye to their bottom line, there are other intelligence benefits to the authorities in Beijing.

Read the full piece here.

Share

Screenshot of videos from a TikTok account linked to a Chinese covert influence campaign.

In a welcome development last year, TikTok announced that it would start publishing insights about the covert influence operations it identifies and removes from its platform globally in its quarterly community guidelines enforcement reports.

Since then, the platform has published three quarterly reports that identified 22 separate covert influence operations originating in countries as various as Russia, Azerbaijan, Ireland, Georgia, Kenya, and Taiwan. But there has been one glaring exception: China.

In my latest piece in The Strategist, I suggest that the Australian government’s proposed Combatting Misinformation and Disinformation Bill 2023 might put TikTok on a path towards greater transparency.

Read it here.

Subscribe now

Source: @elonmusk

Commentary on the government’s draft of the Combatting Misinformation and Disinformation Bill 2023 has reached a fever pitch.

The bill sets up Canberra to be a ‘back-up censor’ ready to urge the big tech companies to ‘engage in the cancellation of wrong-speak’, wrote Peta Credlin in The Australian under the headline ‘“Ministry of Truth” clamps down on free expression’. For Tim Cudmore writing in The Spectator, the bill represents nothing less than ‘the most absurdly petty, juvenile, and downright moronic piece of nanny-state governmental garbage ever put to paper’.

In reality, the intentions of the bill are far more modest than the establishment of a so-called Ministry of Truth. Indeed, they’re so modest that it may come as a surprise to many that the powers don’t already exist.

In my latest piece, published in The Strategist, I explain why everyone needs to calm down. Read it here.

A boat on a sea of ones and zeros computer code that looks like The Great Wave off Kanagawa, digital art. / DALL-E

I’m heading up to Parliament House today to take part in the Senate Select Committee on Foreign Interference through Social Media. The following article on TikTok was my contribution to ASPI’s submission to the inquiry. The full version is available for download on the Parliament of Australia website.

Subscribe now

There are 3 main national security risks with the PRC-owned video-sharing app TikTok that Australians should be concerned about. Two of them—data and content manipulation—are applicable to most other major social media apps regardless of their country of origin. The third risk, that a single political party, the Chinese Communist Party has decisive leverage over TikTok, exacerbates the former two risks and is unique to TikTok as a major mainstream social media app.

The first, and most discussed risk is about data. Following years of scrutiny, TikTok has been forced to be more forthcoming about the fact that TikTok user data is accessible and has been accessed from the PRC. Close observers of TikTok statements from as early as 2020 know that it has only ever been TikTok's goal for China-based employees to have minimal access to user data–not to cut it off completely.

Furthermore, the app relies on this access to function. As stated in a September 2020 sworn affidavit by the company's then Chief Information Security Officer, ‘TikTok relies on China-based ByteDance personnel for certain engineering functions that require them to access encrypted TikTok user data.’

In 2023, this still has not changed. Even as the company puts into place its US$1.5 billion plan dubbed ‘Project Texas’ to move all data attached to American users to the United States, and to institute various governance, compliance and auditing systems to mitigate national security concerns, TikTok vice president Michael Beckerman maintains that engineers based in China "might need access to data for engineering functions that are specifically tied to their roles." At a Senate hearing about social media and national security in September 2022, Vanessa Pappas TikTok’s chief operating officer, declined to commit to cutting employees in China off from the app’s user data.

 As long as PRC-based engineers are able to access TikTok user data, that data is at risk of being accessed and used by PRC intelligence services. TikTok's constant refrain that user data is stored in Singapore and the United States and that it would never hand over the data to the Chinese government even if it were asked is beside the point. The location in which any data is stored is immaterial if it can be readily accessed from China. 

Moreover, TikTok’s parent company, ByteDance, couldn’t realistically refuse a request from the Chinese government for TikTok user data because a suite of national security laws effectively compels individuals and companies to participate in Chinese ‘intelligence work’. If the authorities requested TikTok user data, the company would be required by law to assist the government and then would be legally prevented from speaking publicly about the matter.

Unfortunately, even if TikTok's parent company ByteDance were able to sever access to the app's user data from the PRC, Beijing's intelligence services could still readily access sensitive data on virtually anyone in Australia via the commercial data broker market. 

Second, in what has unfortunately been an under-discussed risk, TikTok could continue to skew its video recommendations in line with the geopolitical goals of the Chinese Communist Party. This is a threat that continues to worsen as more and more people get their news and information from online platforms such as TikTok over which the Chinese party-state can control, curate and censor content.

There is ample evidence that TikTok has done this in the past. Leaked content moderation documents have previously revealed that TikTok has instructed “its moderators to censor videos that mention Tiananmen Square, Tibetan independence, or the banned religious group Falun Gong,” among other censorship rules. TikTok insists that those documents don't reflect its current policy and that it had since embraced a localised content moderation strategy tailored to each region.

In ASPI's 2020 report into TikTok and WeChat, we found they suppressed LGBTQ+ content in at least 8 languages. After British MPs questioned TikTok executives about our findings, they publicly apologised. Our report also included a deep dive on TikTok’s Xinjiang hashtag & found a feed that was flooded with glossy propaganda videos with only 5.6% of those videos being critical of the crackdown on the Uyghurs.

In 2022, TikTok blocked an estimated 95% of content previously available to Russians, according to Tracking Exposed, a nonprofit organization in Europe that analyzes algorithms on social media. In addition to this mass restriction of content, the organisation also uncovered a network of coordinated accounts that were using a loophole to post pro-war propaganda in Russia on the platform. In other words, at the outset of Putin's invasion of Ukraine, TikTok was effectively turned into a 24/7 propaganda channel for the Kremlin.

Following years of intense scrutiny, it is unlikely that TikTok will, in any overt way, become a conduit for pro-CCP propaganda. In a welcome sign in recent months, the company has even begun to label "China state-affiliated" accounts on the platform. It is unclear if these labels also ensure that the content is reduced on the platform as it currently does on other platforms like Twitter.1 To further build confidence, TikTok should, as other social media platforms have, regularly investigate and disclose information operations being conducted on the platform by state and non-state actors.

Any manipulation of the public political discourse on TikTok is likely to be subtle. Unfortunately, because each user's TikTok feed is different, any influence the CCP has over the app will be very difficult to track. It would be trivially easy for the app to, for example, promote or demote certain political speech in line with the CCP's preferences. The app could tip the scales in favour of speech attacking a political candidate who is critical of the  CCP, for example. 

TikTok certainly has the ability to detect political speech on the app as it monitors keywords in posts for content related to elections so that it can then attach links to its in-app elections center. Experiments conducted by nonprofit group Accelerate Change found that including certain election-related words in TikTok videos decreased their distribution by 66%. They also found that TikTok is consistently suppressing videos when it can detect they are about voting.

In 2020, U.S. TikTok executives noticed views for videos from certain creators about the U.S. presidential election were mysteriously dropping 30% to 40%, according to people familiar with the episode and cited by the Wall Street Journal. After making enquiries, the executives found out that a team in China had made changes to the algorithm to play down political conversations about the election.

Algorithmic manipulation of content is not limited to TikTok. To take one recent example in February 2023, Twitter chief executive Elon Musk rallied a team of roughly 80 engineers to reconfigure the platform’s algorithm so his tweets would be more widely viewed. There is clearly a need for all social media companies to be more transparent about how changes to their algorithms affect the content users receive.

The third risk, rightly identified by Cybersecurity Minister Clare O'Neil as a "relatively new problem,” is that apps like TikTok are, as the minister put it, "based in countries with a more authoritarian approach to the private sector."

For TikTok's parent company ByteDance, this authoritarian approach has included compelling the company's founder Zhang Yiming to make an abject apology in a public letter for failing to respect the Chinese Communist Party’s ‘socialist core values’ and for ‘deviating from public opinion guidance’—one of the CCP’s terms for censorship and propaganda.

The enormous leverage that the CCP has over the company is what drove the company at the time to boost its army of censors by an extra 4,000 people (candidates with party loyalty were preferred) and it’s what continues to motivate ByteDance to conduct ‘party-building’ exercises inside the company.

In April 2021, Beijing quietly formalised a greater role in overseeing ByteDance when state investors controlled by the China Internet Investment Fund (controlled by internet regulator CAC) and China Media Group (controlled by CCP’s propaganda department) took a 1% stake in ByteDance’s Chinese entity, Beijing ByteDance Technology, giving it veto rights over the company's decisions. At the time, one of the other two seats on the company's board was held by Zhang Fuping (张辅评) who was the secretary of the company's Party Committee.

More recently the CAC named a director from its bureau overseeing data security and algorithmic governance to the board of ByteDance’s main Chinese entity. According to the Wall Street Journal, this director replaced another CAC official who was formerly part of the regulator’s online opinion bureau.

The PRC party-state is, in other words, completely intertwined with ByteDance to the extent that the company, like many other major Chinese tech companies, can scarcely be considered a purely private company that is only geared towards commercial ends. These companies are neither state-owned nor private, but hybrid entities that are effectively state-controlled.

Too much of the public discussion about the risks of TikTok has been narrowly focused on data security. Even if TikTok were to completely sever access to its user data from China (which it does not plan to do), China's intelligence services could still buy similar user data from data brokers.

It therefore would be to Australia's benefit if more rigorous data privacy and data protection legislation were introduced that apply to all firms operating here regardless of ownership. If protecting national security and guarding against foreign interference are our goals, a broad approach such as this is necessary. 

But a complete overhaul of regulation around data will still not address the risk that the CCP could leverage its overwhelming influence over TikTok and its parent company ByteDance in order to manipulate Australia's political discourse in such a way that would unlikely be detected.

There is no technical fix to a problem that is driven by ideology. The CCP considers the country’s lack of soft power or “international discourse power” (国际话语权), as having a “discourse deficit” (话语赤字) against the strength of Western media and governments, which in turn has a serious impact on China’s international ambitions. The party is open about its view that homegrown social media apps like TikTok present the opportunity to leapfrog the West and begin to meaningfully close that gap. In the past they have attempted to conduct their influence operations on Western social media apps in a process referred to as “Borrowing a boat out to sea" (借船出海). With TikTok, they own the boat.

Share Red Packet

“A rocket zooming away from a big blue bird, 3d render./ DALL-E”

Not long after he bought Twitter, Elon Musk tweeted, I can only imagine despairingly, "The (currently) quiet majority should tweet more!"

At the time, the world's richest man was maintaining that the reason he acquired the platform was "because it is important to the future of civilization to have a common digital town square." The exodus of right-wing users to Twitter clones like Truth Social and other echo chambers was putting the status of the bird site as the world's digital town square at risk, Musk claimed. The remaining “quiet majority” had been cowed into silence by the woke mob. Only he could fix it.

Only he hasn't. Instead, we've seen a steady acceleration towards what Cory Doctorow has labelled the "enshittification" of the platform. As Doctorow argues, leaving Twitter for rival upstarts involves high switching costs. The more expensive it is to migrate elsewhere, like Mastodon, “the larger the ration of shit they can serve to their users—more ads, more boosted posts, less moderation, less reliability—without losing those users’ business.”

Clearly addicted to Twitter himself, Musk seems to be banking on the hope that the perverse pull it has on him is a more universal experience that applies to the rest of us. "On Twitter, likes are rare & criticism is brutal, Musk tweeted in 2018. "So hardcore. It's great." It may be a source of constant distraction, anxiety and enragement but you just can't quit it, he seems to be saying. In fact, you probably enjoy it.

Well, despite the benefits I’ve reaped from it over the years, I wouldn’t say I actually enjoy it. In fact, over the years and especially since Musk bought it, I've jerry-rigged an elaborate system for extracting what value I can get out of the site without being sucked into its vortex. I've spent freely on other apps that surface content from it or allow me to post to it, just as long as I don't have to log into it.

I've even bought a lifelong subscription to an app that forces me to either take a deep breath, follow a bouncing ball with my finger on the screen, or take a long hard look at myself for 30 seconds whenever I try to open Twitter. If I spend more than 5 minutes on it, I get a notification to admonish me: "It's been 5 min on Twitter already. Maybe it's time to return to reality?"

The idea that I would pay for the privilege of being visible on a platform that I actively try to avoid is obviously absurd to me. And I suspect that ponying up to pay for Twitter Blue is not high on the agenda for the many Twitter users who prefer to hang back and gawk at the spectacle. Despite Musk's grim view of human nature, the site’s "quiet majority" likely don't tweet because they recognise that Twitter isn't actually the world's town square, it's the world's colosseum, and they're not that interested in being thrown to the lions.

At any rate, Twitter's toxicity has worsened since Musk's acquisition to the point where it's not just the "quiet majority" he has to worry about but members of the chattering class who are desperately trying to abandon ship. That's why I, like many others, joined Mastodon a few months ago—despite the switching costs—and it's why I posted my first Substack Note just the other day:

It turns out the answer to that question is yes, by the way, but not that many. Certainly not as many as the writers who have been steadily building up followings on Substack in recent years.

Which is why you're reading this. It seems that, unlike Twitter where you have to pay to be seen, users are more visible on Substack Notes if they write articles and then attract subscribers who are interested in what they have to say. Some of those subscribers even pay!

So instead of paying Elon Musk $8 a month for the privilege of providing him with free content, I’m going to write here. I might even eventually have some people here who want to pay me. For now, if you’re interested in hearing what I have to say, which is usually about China, censorship, surveillance and propaganda, please subscribe.

Subscribe now