How to roll out AI in a creative business
I talked recently about how changing the culture of learning in your business will be important if you will make the most of AI. But no matter what, you’re going to have to roll it out – and you need to do that in a structured way.
Remember, this isn’t just an ordinary technology roll out: it’s a change management process that will have a lot of impact on your business. One framework which can help, and that I have found incredible powerful for managing change at scale is the ADKAR model of change management.
This model consists of five stages: Awareness, Desire, Knowledge, Ability, and Reinforcement. Each stage focuses on a different aspect of the change process, from creating a clear vision and generating buy-in, to acquiring the necessary skills and (importantly) sustaining the change over time, something that’s often neglected.
So how might you use ADKAR when looking at an AI rollout?
Awareness
At this point, your focus is to communicate the need and benefits of AI for your business, such as improving efficiency, enhancing customer service, or gaining insights. Explain how AI aligns with your vision, strategy and values, and what challenges it can help you overcome. Use data and evidence to support your case and address any concerns or misconceptions.
Remember, too, that this stage is about the need for change, not that change is happening. The most important outcome for this stage is that everyone understands the “why”.
Key elements of building awareness
- Start with your senior leaders. In effect, you need to go through a managed change process with them first, to ensure they are all aware of the need for change, have a desire to implement it, and have the knowledge they need to do so. Your senior team has probably been through quite a few changes, but none of them will have gone through what you are going to experience with AI.
- Explain the business drivers making the use of AI essential. Don’t sugar coat this, but be mindful of not using “doom” scenarios. Your model should be Bill Gates’ “Internet Tidal Wave” rather than Stephen Elop’s “Burning Platform”.
- For every single communication, ask yourself whether it contributes to helping employees be able to think "I understand why this change is needed". If not, rethink that comms.
- Be clear and consistent in messaging – and have leaders deliver the message (but make sure they are clear about it themselves).
- Tailor your message. Customize communication for different groups within the organisation. Different stakeholders may have different concerns and questions, so addressing them specifically can be more effective.
Desire
Building desire is all about cultivating willingness to support and engage with the change, and for AI, it’s incredibly important. While AI is a technology, it requires cultural change to succeed – and changing a company culture is very hard. Without building desire, any change which threatens the existing culture will fail.
There are many factors which influence whether you can create a desire for change. Personal circumstances will matter, and the fear with AI is that employees will lose their jobs. That’s a big barrier to building desire.
And, in some cases, those fears will not be misplaced, so it’s critical to be clear about your plans if you are to win enough trust to create desire. Consider, for example, making a commitment to reskill employees whose roles are affected by AI, rather than giving bland statements about avoiding redundancies “where possible”.
This is especially critical if you have a poor track record of managing change – so it’s vital that you are in touch with how your change management record really looks to your teams.
At this point, you should also identify your champions. Who, in the business, has a lot of influence? Who are the people who are at the centre of many things, who act as communicators? Who do other employees go to for help and advice? Are there people who, when a new project starts, are the first names on the list? They are not always senior, so make sure you’re looking across the board for your champions.
Even if they are not the most senior people or the most engaged with AI at this point, if you win them over and make them part of the project, you will reap the benefits.
Remember, too, that desire is personal to everyone. While making the business more efficient and profitable tends to get your senior team grinning, not everyone in your business is motivated by that. Focus, too, on the benefits for people’s careers, work/life balance, and especially with AI, freeing up time to do more creative things and less routine work.
And don’t, whatever you do, talk about how “if we don’t become more efficient, people will lose their jobs”. I’ve seen this approach taken many times, and in creative businesses, it almost never works. Desire is about motivating people to change, and fear is a bad motivator.
Key elements of building desire for AI:
- Inspire and engage your team members to participate in the AI adoption process.
- Identify and involve key influencers and champions who can advocate for AI and influence others.
- Highlight the personal and professional advantages of AI, such as learning new skills, increasing productivity, or advancing career opportunities.
- Create a sense of urgency and excitement around AI and its potential.
Knowledge
If awareness is about the why, the knowledge stage is about the how: how are we going to use these tools? This is where you build knowledge of the tools and the processes by which you use them.
One mistake that I have seen made – OK, to be honest, I have made – is to focus too heavily on training people on how to use a tool, without also training on changes in the processes you’re expecting people to make. Every new tool, including AI, comes with processes changes. And, in fact, the process changes that the tool enables are where you achieve the biggest benefits.
Training people in the context of the processes they follow (and any associated changes) relates the training to what people do – and that’s why I would recommend role-based training, which may cut across teams. If you have large teams, consider further segmenting this according to levels of experience. But I would recommend that you train everyone if possible: people who are left out may end up feeling either that AI isn’t relevant to them (and it will be) or that they have no future in your new, AI-enabled business.
Key elements of building knowledge of AI:
- Provide adequate and relevant training and resources for your team members to learn about AI and how to use it effectively. Make sure you document any process changes.
- Tailor the training to suit different learning styles, levels of expertise, and roles.
- Use a range of methods, such as workshops, webinars, online courses, or peer coaching.
- Encourage feedback and evaluation to measure progress and identify gaps.
Ability
So far, what we have done is all theory. This stage is where the rubber really hits the road because it’s where all that training starts to be implemented. And at this point, people will start to spot issues they didn't see before as they get the hang of new processes and get better at them. They will also find things you didn’t anticipate, and even better ways of using AI.
One aspect that’s critical at this stage is the generation of short-term wins. For a lot of your teams, AI is the proverbial big scary thing which is going to cost them their jobs – and even if you have had a successful “desire” phase, it can be easy for people to be knocked off course when that is at the back of their minds, or they are reading scare stories about how AI will mean the end of humanity.
Quick wins will help with this. They are positive, visible evidence about the success of people they know using AI, and in storytelling terms that is absolute gold dust. Remember, though, that the positives must be personal, and in a creative business they need to focus on improving the creative work. Shaving 10% of the time taken from a boring business process might be incredibly valuable to you, but it’s not all that compelling to a writer, editor, or video producer.
Key elements of building ability in AI:
- Support your team members to apply their AI knowledge and skills in their daily work.
- Create a safe and supportive environment where they can experiment, practice, and learn from mistakes.
- Provide guidance, feedback, and recognition to reinforce positive behaviours and outcomes.
- Make sure success stories are being shared, and that your teams are helping each other.
- Monitor and track performance and results to ensure quality and consistency.
Reinforcement
This stage focuses on activities that help make a change stick and prevent individuals from reverting to old habits or behaviours, and I think it’s both the most crucial stage of managing a change in technology or process – and the one that’s easily forgotten.
There are several reasons for this. First, commitment even among your senior team may be waning, leading to reduced encouragement from the top to continue along the path. The people who thought that your rollout of AI was likely to fail will probably be latching on to every bump in the road and turning them into roadblocks – ones that they “knew would happen”.
This is why it’s incredibly important to have all your senior team go through a parallel managed change process, to make sure they are all bought into what you want to achieve. AI is a strategic change on the same level of impact long-term as a complete restructure of your entire business, so there is no getting round managing that process for your senior team.
If you are starting to get resistance to AI deployment at this stage, check whether your senior team are still bought into it. In the worst case, some of them may be sending subconscious signals to their teams that they don’t have to keep going.
And now the bad news: in terms of budget, the reinforcement phase may cost as much as the training required in the knowledge phase because you need people looking after the AI roll out who are constantly engaging with your teams, understanding issues, celebrating success, and making sure that communications about how AI works for everyone, and – importantly – keeps everyone updated on new developments and changes.
For every new pitch, product or process, someone needs to be in the room asking how you can use AI to improve this, speed it up, or do interesting creative things. That is the only way they AI will become embedded in what you do, and not fade away – as so many corporate projects do.
Who is that person going to be? The likelihood is that in the “desire” phase, internal champions will emerge who can do that job. This offers the advantage of credibility, as it’s someone who is both personally familiar and professionally respected, but don’t make the mistake of assuming this role is something that you can tack on to a day job. Unless your business is very small, doing all this is a full-time role, for at least a year after you have “completed” the rollout of the technology.
Key elements of reinforcing AI use:
- Celebrate and reward your team members for their achievements and contributions to the AI adoption process.
- Focus on improvements in employees’ experience, not just business benefits.
- Solicit and act on feedback to improve and refine your AI practices and policies.
- Reinforce the benefits and value of AI for your business and your team.
- Keep your team informed and updated on the latest AI trends and developments and encourage continuous learning and improvement.
AI is about people, not just machines
It would be a little remiss of me if I didn’t mention the launch this week of Microsoft’s consumer and small business AI play. Microsoft Copilot Pro integrates with Word, Outlook, PowerPoint, and OneNote, and offers suggestions, corrections, and insights based on the context and purpose of the document. It’s available now for anyone with a Microsoft 365 Personal or Family plan, at – by complete coincidence, I’m sure – almost the same monthly price as ChatGPT Plus.
Microsoft's approach with Copilot Pro and other AI services is primarily aimed at enhancing practical business and personal productivity, rather than implementing radical changes in the ways people work.
For me, the real short-term win from Large Language Models lies in their ability to clear away yawn-inducing office tasks. Copilot, especially, is a superstar at this. It helps you tackle all the routine stuff, and leaves people to get on with the creative work.
The people factor
AI tools like Copilot are not magic bullets that can solve all our problems, and they don’t magically do things on their own. And that highlights something that I suspect is getting neglected: long-term training and support for users in businesses.
One thing I have noticed in the creative industries time and again: technology often gets side-lined in learning and development. Sure, publishing companies have massively improved when it comes to fostering skills in leadership, coaching and other business areas. Yet, when it comes to embracing and learning new tech, training tends to be pretty old-fashioned, rolled out in a “one and done” approach. Updates are relegated to the odd email (which no one reads).
In the old days, that worked because the pace of change of technology was comparatively slow. A new version of QuarkXpress (yes I am that old) would come out every couple of years, you would do an update session and that was it.
But for cloud technologies this is not enough, and when there is a complete paradigm shift in tech – as we’re experiencing with AI – it risks putting you well behind more agile businesses.
According to a report by Oliver Wyman Forum, there is a significant gap between the skills employees believe they need training in, such as AI and big data, creative thinking, and leadership, and the training that employers are currently offering. 57% of employees think the training they are getting isn’t sufficient. And I think they’re right.
Of course, you can implement short-term fixes. But this is also a good opportunity to set up the way you train and the way your people learn for the long term. The next three to five years are going to see the pace of change accelerate, and you need to adapt the systems which allow your people to learn.
Continuous structured learning
Integrating AI tools into your team's workflow isn't a one-time event, but rather a journey of continuous learning. Begin by setting up a framework for ongoing training and support. This could mean setting up regular training sessions to providing access to online courses, interactive tutorials, and detailed manuals. It's not just about the initial learning curve; it's about keeping the knowledge fresh and relevant.
To foster a culture of continuous learning, encourage your team to see AI as an evolving toolset, one that offers new opportunities for growth and innovation. Promote an environment where experimentation is the norm, and learning from mistakes is valued. This approach helps to maintain a level of curiosity and enthusiasm for what AI can bring to the table.
Remember, the key to continuous learning is collaboration and knowledge sharing. By encouraging your team members to share their experiences and insights gained from using AI tools, you create a knowledge-rich environment. Regular team discussions, workshops, or even informal chat sessions can be great platforms for this exchange of ideas.
Not everyone is going to want to get on board. To get tech-hesitant people excited about AI, relate it to their interests and show how it simplifies work or hobbies. Demystify AI with jargon-free explanations and introduce them to easy-to-use tools through hands-on sessions. Sharing success stories of others who've overcome similar fears can motivate them. Ensure support is available for any questions, making their AI journey smooth and approachable, while focusing on its practical, real-world applications.
To put this into action, consider scheduling a monthly 'AI day' where team members can share new findings, discuss challenges, and brainstorm on how to better integrate AI into your workflows. Think about establishing a mentorship program where more experienced team members can guide others through learning about AI. And finally, make sure you are making use of your best communicators, not just the people who are really enthusiastic about AI.
Apple's 27 per cent tithe
Apple has also confirmed that it will charge a commission on purchases made through alternative payment platforms. This commission will be 12% for developers who are a member of the App Store Small Business Program and 27% for other apps.
The commission will apply to “purchases made within seven days after a user taps on an External Purchase Link and continues from the system disclosure sheet to an external website.”
Apple had a chance to turn a legal defeat into a long-term victory. With Google charging 26% in the same circumstances, the company could have adopted rules which dramatically reduced the levy it wants to take, say to 12% for all developers. This would have gained the company a lot of credibility over the long term.
But no. Instead, it chose to protect short-term revenue, and do something which looks petty, hostile to the developers who have made iOS a successful platform, and which will probably end up in court, again.
When it comes to antitrust, perception matters. And sadly for them, in that area, Apple never misses an opportunity to miss an opportunity.
Weeknote, Sunday 14th January 2024
This week there were some exciting work-related developments which I can't really talk about yet, but I might be able to in the future. Or I might not. I know, I am such a tease. I've also been talking to some people about doing some training for them, and we shall see where that gets to.
We caught the last day of Mark Leckey's In the Offing at the Turner Contemporary and I didn't think that much of it. I like Leckey's work generally – Fiorucci Made me Hardcore is brilliant – but this was a collection of other people's work, largely musicians, and it didn't hang together for me. I said afterwards that it is always interesting when musicians dive into visual art because they often don't have the chops to step beyond the original concept. That makes the work a bit one dimensional, and that's how I felt about most of the work here. It falls into the "glad I saw it, but didn't really work" bucket.
Other than that, this has been a quiet week, when I've spent far too much time in the house – perhaps unsurprising given that it's been freezing when it's not been raining (and occasionally both freezing and raining).
The three things which most caught my attention
- Peter Capaldi bemoans the fact that it's much harder for young working class people to get a start in the arts and creative industries. And he's right. I was lucky enough to be part of the tail-end of free education (I paid no fees and had a grant all the way through to postgraduate level) and I am utterly horrified by how education has change. Fuck, as they say, the Tories.
- Neil Gaiman's memorial lecture for Douglas Adams is a good one. I'm amazed I hadn't seen this until this week.
- Hilariously, it looks like there is a massive bot problem on Twitter, as shown by the number of failed AI prompts.
Things I have been writing
I have moved my email from Substack to Wordpress, which means you'll need to subscribe to get my posts. I'm still working out how this works -- I suspect subscribers will get early access, but I'll make them open a couple of days later. The first one is up and it's about the continuing challenge of a return to office.
Things I have been reading
Neal Asher is a science fiction writer that I have been reading since his first novel, Gridlinked. He's an interesting blend of SF and horror which, in his earlier work, absolutely hit the spot. I didn't enjoy his last one that much,but Jenny Trapdoor already feels like a bit of a return to form. The proof will be in the finishing…
The continuing challenge of return to office
This is the first post-Substack edition of my newsletter, the first one delivered via WordPress. At some point in the coming days, the Substack version will be going away completely. If you want to know why, then you might want to read my post on Substack and platform risk, and then have a look at Platformer’s post on why it’s leaving Substack.
It’s just over twenty years since the last time I worked completely in an office. In every job, I have had since 2003 when I left MacUser, I have spent at least a day a week working from home. And even before then, working away from the office was so frequent that I doubt there was a week between 1995-2003 when I wasn’t out for at least a day.
Perhaps that’s one of the reasons I find the apparent desperation to get employees back into offices so strange. According to a survey by ResumeBuilder, 90% of employers plan to return to office by the end of 2024. Over a quarter of them intend to threaten employees who don’t want to return to office with being fired.
But return to office hasn’t been plain sailing. The latest company to come a cropper with its plans is Internet Brands, the parent company of WebMD, which created a video “encouraging” its teams back into the office, ending in a screen – and I am not making this up – which notes “we mean business” and “don’t mess with us”.
Cue much hilarity from the internet. Internet Brands is probably lucky that Twitter isn’t the force it once was because the pre-Musk social network would have run with this one for several weeks. Now, it’ll get buried under an avalanche of rubbish on Threads after a couple of days, or be seen by the couple of million users on Bluesky.
But I digress.
There are many arguments over the effectiveness of remote working vs in-office, some good and some bad. Having a separation between home and work is a good thing, for one, and although older employees tend to have plenty of space to make that work, younger ones in shared accommodation or living with parents often don’t. And younger employees also benefit from working closely with colleagues, who can mentor them informally much more easily when physically present.
I think this is particularly true for the creative industries. Creative people typically like to think of themselves as the definitive solo fliers, coming up with great ideas and then hammering them into shape, before having them ruined by an editor.
But the reality is that creative work is always collaborative. For example, in digital publishing, your content teams and audience development people need to mesh and work as one team; otherwise they can make the human equivalent of the sound gears make when grinding against each other.
And that is why broad directives about the amount of time which people spend face-to-face vs remote are damaging not only to the people involved, who inevitably feel robbed of autonomy and disempowered, but also to a creative business.
To find the most workable approach, leaders should focus on four factors: the needs of the work, the needs of the people, how work gets done, and the new managerial muscle required to manage a hybrid workforce.
Broad, uniform directives can only be effective if all work is identical and performed by a uniform workforce, which is not the case in the creative industries, and not true for most others. Rather than imposing directives, CEOs should empower their managers, particularly those at the front line, to develop a comprehensive understanding of their team's work, the working styles of their team members, and the most effective ways to accomplish their tasks.
Of course that depends on having empowered, well-trained leadership down to the small team level – but you are already doing that, aren’t you?
Finally, A Grocery Cart That Can Save Me From The Horror Of An Ad-Free Moment Of Existence
This is one of those new technologies that’s useful primarily as a viewfinder on a dismal present and a future determined to be even more miserable. Nobody anywhere will like the smart carts. Nobody, anywhere, will find them not-obnoxious. Everybody who does more than a couple of moments of thinking about it will be horrified by the idea of humanity digging gigantic devastating holes in the ailing planet and mining out its contents for the purpose of putting tablet computers onto grocery carts so that they can perform a service repulsive to literally everyone. Nobody—nobody nobody nobody!—wants to live in a society characterized by inescapable omnipresent advertising for consumer products; no one yet born has yearned to have video advertisements take up ever more of their field of vision.
This is one of those paragraphs that I wish I had written
Weeknote, 7th January 2024
I was going to write a sort of yearnote last week, summing up 2023. I didn’t, in the end, mainly because I felt too tired of 2023 to want to write it. It was not the greatest of years of my life. In fact, I would put it down as probably the worst, all things considered. Even lockdown, with all its attendent insanity, had good moments. 2023 really doesn’t feel like it had many.
The first week of the year is always an odd when when you’re working for a company. It’s a time when quite a few people are going to be away, taking the chance of an extended break (if you time it well, you can basically get three to four weeks off).
It’s a little different when you’re not working. The danger is there is no “first week back” in the same way there’s no “last week”. The time can all blur into one miasma of semi-productive, semi-unproductive days. Which then turn into weeks.
Similarly the division between weekdays and weekends becomes something you have to actively preserve, rather than simply relying on a lack of work to magically create your weekend.
And it’s fair to say this week saw a slow start. But towards the end of the week, things picked up. I worked through what I wanted my goals to be for this quarter (and in fact this year). I also started testing Motion, which is an interesting task/calendar app which I like the look of.
What Motion does is automatically create blocks of time for your tasks. You prioritise them, put in deadlines and start dates, and an estimate of how long it will take to do the task. For bigger tasks which take longer than an hour, you can define “blocks” – the shortest block of time that you want to work on that task. Motion then goes off and puts the blocks on your calendar, effectively scheduling in all the work that you have to do.
It sounds more complex than it it, but the end result is pretty compelling: a complete schedule which lets you be clear at all times about what’s next. I started working with it at the end of the week and absolutely blasted through a bunch of stuff which had been lingering on my task list for ages.
Motion started off life as a calendar application and it still works well as one. It will connect to Google, iCloud and Microsoft (work and personal) calendars, and arranges your tasks around meetings. If a new meeting drops in, it will reschedule your tasks, and alert you if suddenly a task becomes impossible to do beyond its deadline.
It’s not cheap: $19 a month if you pay annually, or a stupid $34 a month if you go monthly, I probably won’t subscribe at this point, as I need to watch where my money is going, but I would definitely be tempted in the future.
Christmasnote, 25th December 2023
Technically, I should have written this yesterday, but somehow Christmas Eve feels like an even less appropriate time to be writing than Christmas Day.
Christmas coincides with my birthday, something that’s been a bane and a boon over the years. On one hand, it opens the opportunity of a REALLY BIG PRESENT as a joint Christmas and birthday one, and one I can open a few days before everyone else gets theirs. Although as a child, there were always a few aunties and uncles who managed to sneak in the same present they would have bought me for Christmas anyway.
On the other hand, it’s always meant that my birthday was overshadowed by the big day itself. In my teens and early 20s, this didn’t matter: my birthday was just an excuse for friends to start the Christmas week’s drinking early.
But as friends and family got older, my birthday has always tended to get forgotten, or pushed to one side. Happily, that has coincided with getting old enough that each passing year is treated with as much suspicion as optimism. Your best days might be ahead of you, but there is a hell of a lot less of them than you would like.
Getting old also means that the old acquisitive urges lessen, too, making present buying for me difficult and less obvious. Kim chose wisely, of course, because she puts a great deal of thought into it.
Hence, spending my birthday evening somewhere near Stratford, seeing ABBA Voyage.
I’ve always loved Abba. Old enough to remember when their singles were new and much-anticipated, even as a child I loved the melancholy in their songs. Abba have two modes: full on joy (Dancing Queen) or heartbreak (Knowing me, Knowing you). No band in the world has ever managed to hit both modes so well.
I’ve argued in the past that Abba’s greatest hits are better than The Beatles, and I still think that’s true. There’s something to be written about how Abba are queer in a way that The Beatles never could be. But that’s another essay, and one that I am probably not entirely qualified to write.
Of course, Voyage is holograms: but after about five minutes, you stop thinking about that and just enjoy it. And it’s wonderful: genuinely the best “live” show I’ve seen. It might have made hundreds of millions of pounds, but you can see where the money went to set it up. Even the building was built specially for the show.
I cried. It’s a mark of the breadth of music that I like that the last band I saw live that made me cry was Van Der Graaf Generator, when they reformed in 2005, and I went to their first show.
But perhaps there’s a connection. I cried at VDGG because they were a band I had loved, who I never thought I would have the chance to see. Abba, despite the physical absence of the protagonists, felt in some way the same.
Anyway, if you get a chance, go and see it. It’s good.
Substack and "platform risk"
It’s quite ironic that I spend part of my professional life warning publishers and content creators about platform risk while publishing a newsletter on Substack.
Platform risk can be summed up as the exposure you have to the whims and decisions of platforms on which your business depends. The example that I use a lot is Google: when your revenue depends on mass scale traffic from Google’s organic search results, changes in the algorithm can wipe you out overnight.
But there are other platform risks too. Twitter is a good example of this. For years, journalists and publishers cultivated their individual and brand presences on the social network, not because it delivered huge amounts of traffic (it was low single digits for many brands) but because it was a good place to establish a brand. It was disproportionately used by media people, and so had significant second order effects: you got the attention of people who could bring you more attention via other channels.
That and, as I always say, journalists are gossipy little creatures and Twitter was the world’s biggest journo watercooler. Your audience might not have been on there, but your friends and rivals were.
That’s why Twitter’s implosion under Elon “don’t give them their money back” Musk is so painful for many in media. A lot of independent creators, in particular, have been caught between not wanting anything to do with a platform where abuse of various kinds is encouraged rather than just tolerated, and wanting to protect their livelihood, which depends on connections on Twitter.
Which leads me on to Substack, and the overnight response of its executives to a letter from over 200 users of the service asking it not to host content by Nazis. There were many possible communications approaches the Substack execs could have taken. They appear to have chosen the worst of all worlds.
I’m not going to go into too much detail about why their response is bunk. Ken White has a good summary, and Nick Heer also has a good take. I will, though, note that Substack’s attempt to portray itself as a “haven for free speech” means “speech we decide is acceptable”: the platform does not allow porn, which is protected free speech in the US. Clearly the company is applying its values to decisions about what to host. Porn is bad, but Nazis – actual, real, Nazis – are fine.
Substack is allowed to apply its values to its business. But that also applies to individuals. Freedom of association is as foundational a right as free speech. In fact, arguably, more so. Without freedom of association, freedom of speech is (literally) so much hot air.
Of course, it applies to me too. And one of my core values, really quite a simple one, is not contributing to giving a platform to racists and fascists. Literal Nazis definitely fall into that bucket. By being here, even with a small audience, I am saying I am OK with platforming Nazis.
I am not, for the avoidance of doubt, OK with platforming Nazis. And I don’t want to do business with those that are.
So as of some point between Christmas and New Year – the traditional period for changing tech – I’ll be moving my Substack and mailing list to Wordpress.com, which I already use to host this site. WordPress has features for mailing lists, including paid subscriptions, and it’s easy to move your email addresses over there. Of course, I need to work out how to migrate existing content too.
If you’re currently a free subscriber on Substack, this will be almost completely seamless, except you won’t be able to automatically read my posts in the Substack app (no great loss, in my opinion, but your mileage may vary). You will, though, be able to subscribe via RSS in the Substack app if you really want to.
If you’re an annual paid Subscriber, you’ll be comped a full year starting in January. If you’re a monthly subscriber, you’ll be comped three months (if I can do it – I need to check if that’s possible, but I’ll work something out). As a reminder at present there are no additional benefits to anyone who pays, but I really appreciate that you do.
Ironically, WordPress charges me less of a cut than Substack, because I already host my site there. But if you want to move your own Substack and don’t want to pay a monthly fee, you can do that too – the free plan includes paid emails, at the same rate as Substack.
This move should happen before the new year, and I’ll keep subscribers updated about my progress and any snags I hit along the way. Then, at some point around the middle of January, I will be deleting my Substack, and my account.
Of course, if you disagree with this decision or aren’t comfortable with WordPress.com, please do unsubscribe! Your freedom of association is as important as mine.
Removing cloud storage providers from macOS
One of Apple’s most annoying habits is changing things to make them harder for users to “make mistakes”. Almost without exception, these actually degrade the user experience in one way or another.
A case in point is the changes that Apple made a while ago to how third-party cloud storage providers such as Dropbox, OneDrive, and Proton Drive create and store their files. In the past, these were held in common or garden directories in the user’s folder – so, for example, OneDrive used to store its files in /Users/ianbetteridge/OneDrive. You could even change this location if you wanted to put it somewhere else.
That, obviously, was too simple for Apple. Now, the “approved” way of being a cloud storage provider is to store files in /User/Library/CloudStorage. This uses the macOS File Provider extension rather than custom kernel extensions, and Apple has indicted it will get rid of custom kernel extensions at some point. It hasn’t happened yet, and some storage providers – notably Nextcloud – still use the old method.
The drawback of this is that the CloudStorage director is, like everything in /Library, hidden. You can’t get to it without either holding down option and selecting the Go menu in the Finder or using the terminal. That means if you remove a cloud provider’s drive from the Locations bar in the Finder windows, you have a fun time trying to find those files again.
More importantly, if you remove a provider, it’s hit or miss whether your files will also be removed. I’ve seen instances where deleting a cloud storage provider’s application also asks if you want to delete the locally stored files. But more often, those files will just stick around, in a directory you can no longer easily access, hogging disk space.
That’s fine as long as you have infinite disk space and didn’t locally cache many files. But I like my files local where possible, which means if I delete, say, OneDrive (which has tens of gigabytes of files in it) I just lose that disk space.
However, I hear you say, can’t you just navigate to them and delete the directory? Well, no: Apple (helpfully, again) locks those directories against deletion in the Finder. You can delete the files in them, but the folder itself sticks around like a bad smell. And if you reinstall the cloud storage provider again, things seem to break: when I tried this, the folder refused to sync, and of course, couldn’t just be moved, and if it did appear to sync files didn’t open because the Finder was confused about their sync status.
Thank you, Apple. You have replaced well-written kernel extensions with your own “universal” version, which actually works less reliably. Well done, people.
Thankfully, you can completely delete a folder. You just have to go to the terminal and do a little bit of Unix to do it.
- Open a terminal, and go to the CloudStorage folder – for me, this means typing “cd /Users/ianbetteridge/Library/CloudStorage”, but you will obviously have to replace that with your username.
- Type “ls” – this gives you a list of what directories are in there. For example, I have folders called OneDrive-IanBetteridge and Dropbox, for the two cloud storage providers I use.
- Now the dangerous bit. Type “sudo rm -r NameOfDirectoryToDelete” (replacing, of course, “NameOfDirectoryToDelete” with the name of the folder you wanted deleting). Terminal can autocomplete names, so you can just type the first few letters of a unique name, hit tab, and it will fill in the rest.
- Hit return, and you will be asked for your password. Type it in, hit return again, and that directory and all its contents are gone from your Mac, for good, with no real possibility of getting them back. So really do make sure that you’re deleting the directory that you want.
And that is what I mean about Apple’s ability to make things less usable. If I was able to just navigate to the folder in the Finder and delete it normally, it would be in the Bin and so, if I make an mistake, easier to recover. But because Apple first instinct every time is to “protect” users and “make it easier”, you need to go to the command line and use a Unix tool which is far more user-hostile and dangerous.
Thanks Apple.
Weeknote, Sunday 17th December 2023
I’ve been thinking a lot about how I use computers this week. As I have written before (at great length) I still use computers like I was a full-time technology journalist, which means the tech is the point. I’m playing around a lot, but not necessarily getting a lot done.
So this week, I put a bit of thinking time into both the why and the what. Why I do this is pretty simple: I like playing around with tech, and while I was a full-time journalist, it was literally my job to use it and get to know it better. But what I also had to do was write about it, communicating what I found to other people. I don’t have to do that any more, but I would like to do it again.
The what was thinking about what tools I need to do the productive and creative things that I enjoy doing (and that make me some money). And that list turned out to be simple: a browser, a word processor, a few other things. I’m going to write a longer post about this, but actually sitting down and writing some thoughts on paper about it made me feel a lot calmer.
Three things which you might want to read
- I am shocked, shocked I tell you, that corporations have been using the cover of inflation to pad their margins and add to their bottom line. And I was told it was all greedy workers wanting to, you know, be able to buy groceries.
- No, just because you have paid millions of pounds for a train doesn’t mean you own it. You’ve simply taken on the responsibility of paying inflated fees to the company that made it when it needs servicing. Didn’t you get the memo?
- I really enjoyed John Scalzi’s post about abandoning Twitter. There are some interesting points about engagement on Bluesky and Mastodon compared there, too, if that’s your kind of thing.
What I’ve been reading
One of my bad habits is book bouncing. I’m like a magpie: I start on one thing and bounce to a new, shiny thing. And as I have a massive backlog of books to read, it’s easy to bounce from one to another.
All of which is a long-winded way of saying that this week I have been reading about five books, with no actual substantial reading done. I promise to be good next week.
What I have been writing
Other than this, absolutely nothing. It has not been a productive week.
What the Epic vs Google case means for content
A games company and a search giant tussling over Fortnight doesn't sound like it has much to do with publishing. But it begins a process which could see publishers get more revenue
There has been a lot of controversy over Google and Facebook “stealing” content from publishers. Publishers have been accusing Google of stealing their content for a long time. They claim that Google's search engine and news aggregator are infringing on their copyrights and depriving them of revenue, particularly in a world where more and more answers are appearing on Google’s results pages rather than pushing traffic to publishers.
The argument is that Google is using publisher content without paying them or asking for their permission, and that this is unfair and illegal. They also complain that Google is dominating the online advertising market and squeezing out their competitors. Some publishers want Google to pay them for displaying snippets of their articles or linking to their websites, or to stop using their content altogether.
Google, of course, argues that it is providing a valuable service to both publishers and users. It says that it is helping publishers reach a wider audience and drive more traffic to their websites, while also giving users quick and easy access to relevant and high-quality information. Google also points out that it respects the choices of publishers and allows them to opt out of its platforms or customize how their content is displayed.
Gannett, the publisher of USA Today, is not convinced. It has sued Google for allegedly monopolizing the advertising technology market, arguing that Google's broad control of the ad tech market has hurt the news industry, as online readership has grown while the proportion of online ad spending taken by publishers has decreased.
Gannett isn’t the only one. The Daily Mail alleges that Google has too much control over the online advertising market, which has resulted in newspapers seeing little of the revenue that their content produces. The lawsuit claims that Google controls the tools used to sell ad inventory, the space on publishers' pages where ads can be placed, and the exchange that decides where ads will be placed. The Daily Mail argues that this lack of competition depresses prices and reduces the amount and quality of news available to readers. The lawsuit also alleges that Google “punished” publishers who "do not submit to its practices”.
A lot of this hinges on copyright, and what amounts to effectively extending its provisions to cover something that’s previously been considered fair use: showing snippets of publisher content to let users decide if they want to click through to the content.
Similarly to Cory Doctorow, I would argue that extending copyright like this is foolish, and ultimately won’t benefit publishers and the rest of the creative industries. What will benefit us is removing the choke points that big tech companies like Google and Facebook own, and that world took a significant step forward this week.
Yesterday, a federal court jury ruled in favour of Epic Games in its lawsuit against Google, marking a significant victory for the game developer. The jury found that Google's Android app store, Google Play, uses anticompetitive practices that harm consumers and software developers.
Epic Games had accused Google of abusing its dominant position as the developer of Android to strike deals with handset makers and collect excess fees from consumers. Google collects between 15% and 30% for all digital purchases made through its shopfront. Epic tried to bypass those fees by charging users directly for purchases in the popular game Fortnite; Google then removed the game from its store, which led to the lawsuit.
The jury agreed with Epic on every question it was asked to consider, including that Google has monopoly power in the Android app distribution markets and in-app billing services markets, that Google did anticompetitive things in those markets, and that Epic was injured by that behaviour. What the remedy will be is yet to be determined. But even if it is confined solely to Epic, it’s a step towards breaking the stranglehold that Google and Apple have on the app stores and making the 30/15% tithe they take a thing of the past.
Why does this matter to publishers? Because it opens the possibility of publishers getting more control over the way their content is distributed to smartphones, and ultimately take more revenue by, in effect, running their own app stores just for content.
Being able to have an app store just for your publications would mean an immediate increase in revenue. Currently, between 15- and 30% of every in-app subscription or micropayment goes to the tech giants. If publishers could bypass this fee, they would retain a larger portion of their sales, potentially leading to significant financial gains.
This, as Doctorow highlighted, is where Google and Apple have been taking money from publishers – not by “stealing content”, but by stealing revenue. And it has the added bonus of allowing publishers to get a closer relationship with their customers, something that the app store intermediary model removes. . Customers won’t be buying from Apple or Google and the platform passing on the money to you: they will be buying directly from the people who make the content.
Of course, there are also challenges and risks associated with running an app store. These include the technical and financial resources required to develop and maintain it, ensuring security and privacy for users. Publishers would also have to convince users to switch from established platforms like Google Play and the Apple App Store, which takes more investment in marketing.
Of course, there are downsides. Publishers will need to spend more on discovery because it’s unlikely that either Apple or Google would ever promote a competing App Store. But that’s a small price to pay for restoring the direct relationship you have with customers.
Weeknote, Sunday 10th December 2023
On Tuesday, I attended (virtually, of course) an International Association of News Media (INMA) talk given by Benedict Evans on the future of news. I like Ben’s approach, which is basically to keep reminding news people that putting all their eggs in the basked of Google and Facebook was a bad idea, and no amount of begging them for money is going to make it better.
As Cory often points out, the problem isn’t Google “stealing content” – it’s that Google and Facebook have an effective duopoly over online ads. They are stealing money rather than content. Focusing on MOAR COPYRIGHT isn’t going to fix that.
I spent far too much time this week futzing around with technology rather than doing anything productive with it. Tech is my absolute best (worst) prevarication method. Instead of just getting on and doing stuff with the tech, I spend time farting around with it, installing this, playing around with that. It gives me the illusion of doing something constructive when I’m actually doing nothing of the sort.
One thing I did was to change my contract with Ionos, which I use to host and hold the various domains I have. Back in the old days, I used to self-host WordPress, which I stopped doing when I managed to corrupt an entire database and lose about a decade’s worth of posts. I was still paying for services which I no longer needed, including legacy support for outdated versions of MySQL, so managed to cut down my costs quite considerably. I should have done that sooner.
One project which I might embark on is to trim back my online presences and consolidate into one site. I currently have my tech blog, this site, and also a Substack. Oh, and a small site on Writing.as for short fiction. I’m tempted to merge them all into one, on WordPress, which would be cheaper to run and potentially make more sense.
But I am definitely not embarking on this for a while. Too much other stuff to do.
One project which I really do have to get to grips with is consolidating all my files into a single, coherent place. Every time someone has launched a new online file storage system, I have tried it out. That used to be excusable – it was my job to know about stuff like this – but now it’s not, and it’s in desperate need of consolidation.
I have files on Dropbox, iCloud Drive, and two different OneDrives. There’s a lot of duplication, but the structure of all of them is quite different. It’s going to be a semi-manual mess to work out how to get it all in one place.
And that’s not even thinking about which place it should be. My main rule is that everything must be stored locally on at least one machine, which then gets locally backed up, and as the ThinkPad is the device with the most storage that rules out iCloud. OneDrive seems reluctant to store everything locally, even when I tell it to. That leaves Dropbox.
But that means paying for another storage service, which seems silly when I have a lot of OneDrive storage space. I have a personal Microsoft 365 account, for access to Word/Excel/PowerPoint, and that gives me 1Tb of OneDrive storage, effectively thrown in for nothing.
I have the free version of Dropbox, but because I have had it for a long time and they have done a lot of “get free storage” promos over the years, it gives me nearly 9Gb of free space. That’s enough for a decent-sized “working documents” folder, storage everything that’s in use. Everything else can be be archived easily.
So perhaps that’s a good first step: get everything on to the ThinkPad, which is easily done, sort all the files, and use Dropbox for “working documents”. Sounds like a plan.
But not for today.
Three things which caught my attention this week
- Timothy Burke is a journalist who had his devices seized by the FBI in an investigation into “leaked” Fox News footage. Having paid Google a lot of money over a long period for storage, he’s now been told Google is basically going to close his account and he has seven days to move hundreds of terabytes off their servers, or they will delete it. This is the danger of cloud storage: if you don’t have local copies, your files are not your own.
- Related, Cory wrote about how DRM allows companies to simply force you to accept downgrades to service. Which a bunch of PlayStation users found out the hard way when Sony simply removed content they had bought and paid for.
- Steven Levy wrote about Google’s NotebookLM, which is a service that lets you upload content and use a large language model to query it. This is a smart application of AI, but I want this to be local: I don’t want to have to upload my content into someone else’s servers.
This week I have been reading
Michael Jecks, thriller writer and pen expert (no really) has a new book out – the first that he’s self-published. It’s called One Last Dance Before I Die and as with all Mike’s books, it’s a well-constructed pacey read, which I would highly recommend if you want something fun and light.
I’ve also been reading Richard Skinner’s Writing a Novel, which is pretty good even if he is a bit snotty about genre fiction.
This week I have been writing
Remarkably little because I have been futzing around too much with technology. I did, though, write something last Sunday about resurrecting my MacBook Pro. The only downside I have found to that machine is that, compared to everything else I use, its keyboard really does suck. I’m so glad that Apple went back to proper switches.
Pluralistic “If buying isn’t owning, piracy isn’t stealing”
In Poland, a team of security researchers at the OhMyHack conference just presented their teardown of the anti-repair features in NEWAG Impuls locomotives. NEWAG boobytrapped their trains to try and detect if they’ve been independently serviced, and to respond to any unauthorized repairs by bricking themselves
If you ever needed to see an example of quite how insane the “IP protection” laws are, this is probably it.
The MacBook Pro
The MacBook Pro 16in which was effectively replaced by my MacBook Air M2 has been sitting in a corner for a while. I had wiped it completely – something that’s a bit of a saga in its own right – and intended to sell it.
The buyer I had in mind wasn’t able to take it as unfortunately, they had some financial mishaps, and I would rather not sell it via eBay or classified. So it has just sat there doing nothing.
I decided to set it up and use it for a while, just to remind myself of what it was like. It’s the last generation of Intel machine, and I bought it not long before the announcement of the M-series chips. Although it was the lowest-end of 16in MacBook Pro, it’s still a pretty good computer and it seemed a shame for it to be doing nothing.
I’m glad I did because it’s reminded me how much I like a big screen laptop. It’s nowhere near as good for either performance or battery life as the Air, but it’s still more than fast enough for everything I need to do. And as I am unlikely to stray far from the house with it I don’t need to worry too much about the battery.
Now I just have to think about a name for it because the default “Ian’s MacBook Pro” seems a little bit soulless.
Installing Brew
Whenever I get a new Mac or resurrect an old one, I start from scratch rather than reinstall from a backup. This lets me work out which applications I actually need. Because I like to try out many applications, I end up with a lot of software on my machines which I don’t actually use much.
The days when this mattered from the perspective of system maintenance are long gone. Most applications are spraying extensions, libraries, or even (lord help us) DLLs all over your system. Even Linux has self-contained application installs now, thanks to technologies like AppImage, Flatpak and Snap.
But it’s still a waste of disk space and feels inelegant, so I set everything up with a clean slate and only install when I like using.
One thing that always gets installed on any Mac is Brew, the package manager, which is the de facto standard for installing Unix apps on an Apple computer. macOS is, of course, based on Unix, but the default set up doesn’t include the kind of software which usually comes as standard – utilities like ffmpeg, for example.
You can install them, though, and Brew makes is easy. It’s a command line tool which works in the same way as a regular Linux package manager, like DNF on Fedora or APT on Debian-derivatives. Once you have installed Brew using a single line of commands, you can type brew install
and the name of the software you want, and it will do all the installation you need.
Brew lets you fill the holes which Apple has left. For example, the first thing I install with it is wget
, which isn’t part of the standard macOS and which I find very useful. I also add yt-dlp
so I can download video from YouTube and other services, as well as get_iplayer
to tap into the BBC’s archives.
There’s a lot more you can do with Brew, and if you are used to the command line I recommend it.
Weeknote, Sunday 3rd December 2023
How did it get to be December already? And how did it get so cold?
The Byte archive
I was a fairly religious reader of Byte magazine from the early 1980s until it finally bit the dust as a print publication in 1998. I always loved that it wasn’t focused on a single platform, but on “small computers” as a whole.
It also had the kind of deep technical content which I loved. If you wanted to know about new processors, the transputer, or something even more esoteric, Byte was a great place to keep informed.
It also had Jerry Pournelle. Science fiction writer, conservative, and holder (in later life) of some dubious views, Jerry was nonetheless one of the most influential early computer journalists. I loved his columns, which stretched out to 5,000 words or so per issue. They were written from the perspective of an ordinary computer user — albeit one that had the kind of knowledge required to run a computer in the days of the S100 bus and CP/M.
Thankfully, the Internet Archive has every issue of Byte, scanned and neatly labelled. Annoyingly though, there isn’t a single collection which has every issue in it, which means it’s not easy to just download everything.
And having local copies is vital for me, as I use DEVONThink for research, and it wants a local set of PDFs. So I have started putting together the definitive collection of every single issue, and once I’ve done it I will put them somewhere online, so people can download the whole set. It’s big – my incomplete version is about 7Gb, and I estimate the full set is about 10Gb – but at least they will be there.
This took quite a while to do this week, and I'm pleased with the results.
Chanel
I went to see the Chanel show at the V&A – I don’t really like Chanel’s clothes that much, but her accessories are amazing and she had a really fantastic eye for patterns.
Seeing the collection emphasised that once she created the classic suit, so much of what she did was just more of the same. Milking a hit isn’t necessarily a bad thing: but the small card which notes she “attempted to extend the suit from day to evening wear” is a bit of a giveaway. She didn’t just make more suits. But she was more than happy to keep churning out endless slight variants in a way which made her a lot of money.
It was a little disappointing that the exhibition basically skips over the nine years when she could not work in France because she was widely believed to have collaborated with the Nazis. She had an affair with a German officer and used his connections to protect a relative. Literally one sign, with no more than 30 words on it, and then skipping merrily on to her return in 1954.
In fact, there is more space devoted to the one document from 1943 which lists her as working with the resistance, although there is no documentation (and no one remembers) exactly what she did with them.
There is, of course, far more documentation listing her as a Nazi agent. She definitely benefitted from the Germans’ Aryanization laws, which let her get control of her perfume business from the Jewish Wertheimer brothers.
There’s no doubt that Chanel collaborated, and that her high-placed contacts (Churchill, Duff Cooper, and many others) protected her after the war. None of this is mentioned, perhaps because once you understand what she did and what she was, it’s much less likely that you will just want to admire the pretty clothes.
I don’t think it’s possible to understand Chanel-the-person without considering that period of her life. And the exhibition doesn’t have the excuse that it’s solely about her influence on fashion (there’s surprisingly little which contextualises her in that sense). It ends when she dies, so it's not about Chanel the brand or even really her legacy.
In that sense, it’s a massive contrast to Diva, which is also on at the V&A and which managed to reduce me to tears when I saw it. Diva is a brilliant bit of curation in ways that Chanel is not.
However, if you do go to the V&A, get yourself a piece of the pear and caramel cake. It’s really rather fine.
Three things you should read this week
- The End of Elon Musk. In any rational world, Musk’s performance at the Dealbook conference would be the end of his career. It probably won’t be, unfortunately. But, as Magary notes, Musk “appeared both high and made of plywood”. He does not seem like a well man, and I don’t say that either lightly or with any pleasure.
- Speaking of Musk, the Cybertruck is here1, and predictably it’s pricier and has less range than he claimed. Oh, and while the sides are bulletproof, as Musk said, the windows are not, which may prove an issue if someone is actually trying to kill you.
- And sticking with the theme of “people who really should grow up”, Basecamp lost a customer thank to DHH’s nonsense. Why is it that people who crow loudest about “keeping politics out of work” so often bring their politics to work? Of course, what they actually mean is “keep your politics out of work”. It’s the same as Elon “Free Speech” Musk. Free for them, not for you.
This week I have been reading…
The news that greenhouse gas emissions have been soaring rather than reducing ended the week on a sour note for me, but it makes it more obvious than ever that capitalism isn’t going to deliver a future of humanity. So reading Tim Jackson’s Post Growth has been pretty timely. Highly recommended.
-
Not actually here till 2024, or 2025 if you want the cheap model, and not here at all outside the US because it doesn’t meet any reasonable safety regulations. ↩︎
Open extensions on Firefox for Android debut December 14 (but you can get a sneak peek today) | Mozilla Add-ons Community Blog
Open extensions on Firefox for Android debut December 14 (but you can get a sneak peek today):
Starting December 14, 2023, extensions marked as Android compatible on addons.mozilla.org (AMO) will be openly available to Firefox for Android users.
But not of course for iOS, because Apple doesn’t allow companies to use any rendering engine other than Safari’s webview. And Apple also hates the idea of extensions that aren’t themselves applications, so don’t expect them to make the lives of extension developers easy once the EU forces them to open things up a little.
How to get a glimpse of the post-Google future of search
What does the search engine of the future look like? Forget 10 blue links...
You can break down the creative process into three big chunks: research, creation and revision. What happens in part depends largely on the kinds of content you're creating, the platforms you are making the content for, and many other factors.
Like every journalist, I spent a lot of time using search on the web to help with that research phase. I was quick off the mark with it, and I learned to adapt my queries to the kinds of phrases which delivered high-quality results in Google. My Google-fu was second to none.
But that was the biggest point: like all nascent technologies, I had to adapt to it rather than the other way around. Google was great compared to what came before it, but it was still a dumb computer which required human knowledge to make the most of out of it.
And Google was dumb in another way too: apart from spelling mistakes, it didn't really help you refine what you were looking for based on the results you got. If you typed in “property law” you would get a mishmash of results for developers, office managers and homeowners. You would have to do another search, say for “property law homeowners” to get an entirely different* set of results that were tailored for you.
Google got better at using other information it knows about you (your IP address, your Google profile) to refine what it shows you. But it didn't help you form the right query. It would ask you “hey, what aspects of property law are you interested in” and give you a list of more specific topics.
What's more, what it “knew” about you were pretty useless. You couldn't, at any point, tell it something which would really help it give you the kinds of results you wanted. You couldn't, for example, tell it "I'm a technology journalist with a lot of experience, and I favour sources which come from established sites which mostly cover tech. I also like to get results from people who work for the companies that the query is about, so make sure you show those to me too. Oh, and I'm in the UK, so take that into account."
Google isn't like that now. Partly that's down to the web itself being a much worse source of information. But that feels like a huge cop-out from a company whose mission is to “organise the world’s information and make it universally accessible and useful”. It sounds like what it is: a shrug, a way of saying that the company's technology isn't good enough to find "the good stuff".
The search engine of the future should:
Be able to parse a natural language query and understand all its nuances. Remember how in the Knowledge Navigator video, our professor could ask just for “recent papers”?
Know not just the kind of information about you that's useful for the targeting of ads (yes Google, this is you) but also the nuances of who you are and be able to base its results on what you're likely to need.
Reply in natural language, including links to any sources it has used to give you answers.
If it's not sure about the kind of information you require, ask you for clarification: search should be a conversation.
For the past few weeks, I've been using Perplexity as my main search engine. And it comes about as close as is currently possible to that ideal search engine. If you create content of any kind, you should take a look at it.
Perplexity AI allows users to pose questions directly and receive concise, accurate answers backed up by a curated set of sources. It's an “answer engine” powered by large language models (including both OpenAI's GPT-4 and Anthropic's Claude 2). The technology behind Perplexity AI involves an internal web browser that performs the user's query in the background using Bing, then feeds the obtained information to the AI model to generate a response
Basically, it uses an LLM-based model to create a prompt for a conventional search engine, does the search, finds answers and summarises what it's found in natural language, with links back to sources. But it also has a system it calls (confusingly) Copilot, which provides a more interactive and personalised search experience. It leverages OpenAI's GPT-4 model to guide users through their search process with interactive inputs, leading to more accurate and comprehensive responses.
Copilot is particularly useful for researching complex topics. It can go back and forth on the specific information users need before curating answers with links to websites and Wolfram Alpha data. It also has a strong summarisation ability and can sift through large texts to find the right answers to a user's question.
This kind of back-and-forth is obviously costly (especially as Copilot queries use GPT-4 rather than the cheaper GPT-3.5). To manage demand and the cost of accessing the advanced GPT-4 model, Perplexity AI limits users to five Copilot queries every four hours, or 600 a day if you are a paid for “Pro” user.
If you're not using Perplexity for research, I would strongly recommend giving it a go. And if you work for Google, get on the phone to Larry and tell him your company might need to spend a lot of money to buy Perplexity.
Latenote, Monday 27th November 2023
Between getting ridiculously excited about the goings-on OpenAI, I didn't get a lot of writing done this week. There are definitely times when too much is going on in the tech world, and my old habits die hard: I have to keep up with it all.
I wrote a post on Substack with my take on it, from the perspective of the longer-term impact on creative professionals. And, given how fast things were moving, I ended up rewriting it three times. That was a good reminder not to cover breaking news in that newsletter!
In case you're interested, the focus of that newsletter is the three-to-five year perspective on how technology will impact on what we occasionally call “the creative industries”. That includes magazine publishing, of course, but also writing and creativity more broadly. Hopefully, it should be interesting.
On Sunday, we went out with the wonderful and super-clever Deb Chachra, who has just published her book How infrastructure works (and there's a great review of it here if you are interested). We tempted Deb out of London on a trip to Dungeness, which has both Derek Jarman's cottage and Dungeness A and B nuclear reactors. What's not to like about art and infrastructure?
And more art on Sunday night, as we went down to Folkestone for a talk by the brilliant and wise Jeremy Deller. If you don't know Deller's work, honestly, where have you been for the last 20 years? This is the third time we have done something Deller-related this year, having seen him before in London and also seen Acid Brass. 2023: Year of the Deller.
The three things which most caught my attention
- Commiserations to my old comrades in SEO, who are dealing with some pretty turbulent times. I promise that I didn't sabotage Google.
- Bill Gates wrote a long post about the way AI is going to change the way you use computers. Gates is right – large language models are just the precursor to what might look from some angles like the end of application software altogether.
- Bloomberg looked at the way Elon Musk has been radicalised by social media, adopting a world-view that's completely in the thrall of what we would have called the alt-right not that long ago.
Things I have been writing
There were three… no, actually four drafts of my post about what was going on at OpenAI and why you should care. I am never doing breaking essays on news again.
To give myself a break from all things Orford, I picked up a short story that I had left to one side, about a very strange doctor. Might finish that this week.
What the heck is going on at OpenAI (and why should I care?)
Confused? You should be. I'm deliberately not looking at Techmeme so I don't have to update this post for the fifth time.
Twenty-four hours ago, this was a thoroughly different post. Heck, twelve hours ago, it was a different post.
One of the things I told myself when making this Substack was that I wouldn’t focus on current events. My focus is on the longer term: the three-to-five-year time frame, for publishers, communications professionals and others assorted nerds.
But the shenanigans at OpenAI over the weekend suckered me in, and now I have had to rewrite this post three times (and whatever I write will probably be wrong tomorrow). Still, the drama goes on.
The drama that’s been happening at OpenAI does matter and might be a turning point in how AI and large language models develop over the coming years. It has some implications for Google – which means it is relevant for publisher traffic – and Microsoft – which means it is significant for the business processes which keep everything flowing.
What’s happened at OpenAI?
If you’ve not been following the story, here’s a timeline created by Perplexity (about which I will have more to say in the future). But the basics are that OpenAI’s board dismissed Sam Altman, its founder and CEO, alleging he had been less than truthful with them. Half the company then decided they wanted to leave. Microsoft’s Satya Nadella then claimed Altman would be joining his company, only to walk that back later in the day. Now Altman is going back to OpenAI as CEO but not on the board and there will be an “independent investigation” into what went on, something that might not totally exonerate Altman.
Confused? You should be. Everyone else is. Partly this drama comes down to the unusual structure of OpenAI, which at its heart is a non-profit company that doesn’t really give two hoots about growth or profits or any of the things most companies do. Partly it’s down to Altman basically pushing ahead as if this wasn’t true, then realising too late that it was.
What’s the long-term impact on future AI development?
OpenAI has been at the forefront of developing the kind of conversations large language models which everyone now thinks of as “AI”. It’s fair to say that before the June 2020 launch of GPT-3, LLMs were mostly of interest to academic researchers rather than publishers.
And a huge number of tools have been built on top of OpenAI’s technology. By 2021 there were over 300 tools using GPT, and that number has almost certainly gone up an order of magnitude since. And of course, Microsoft is building OpenAI tech into everything across its whole stack, from developer tools to business apps to data analysis.
If there’s one company that you don’t want to start acting like a rogue chatbot having a hallucination, it’s OpenAI.
And yet, because of Microsoft’s investment in the company and commitment to AI, it probably matters a lot less than it would have if this schism had happened three or four years ago. The $13bn it has put in since 2019 for an estimated 49% stake of the company and the fact it is integrating OpenAI tech into everything it does mean it has a lot to lose (and Satya Nadella does not like losing.)
Because of this, I think the past few days won’t have much impact on the longer-term future of AI. In fact, it could end a good thing, as it means Microsoft has committed that it will step in should OpenAI start to slip.
The greatest challenge for Microsoft was that, although it had perpetual licenses to OpenAI’s models and code, it didn’t own the tech outright, and it didn’t have the people in house. And, when you’re betting your company’s future on a technology, you’re always in a better position if you own what you need (something that publishers should take note of).
Partners are great, but if you’re locked into a single partner, and they have what you require, you’re never going to be the driver of your fate. Now, though, if Altman and the gang join, Microsoft effectively owns all it needs to do whatever it wants. It has the team. It has the intellectual property. Everything runs, and will continue to run, on Azure, and it has the financial muscle to invest in the huge amount of hardware required to make it available to more businesses.
The big question for me is how all this impacts on Google over the next few years. If Altman and half of OpenAI ends up joining Microsoft, I think it weakens Google substantially: at that point, Microsoft owns everything it needs to power ahead with AI in all its products, and the more Microsoft integrates AI, the stronger a competitor it will be.
If, on the other hand, Altman goes back to OpenAI with more of a free hand to push the technology further and harder, Microsoft still benefits through its partnership, but to a lesser degree.
If I was running Google, I would be calling Aravind Srinivas and asking how much it would take to buy Perplexity. But that’s another story, maybe for next week.