One person on our creative team tests the AI tools. Everyone else ships.

For a few months now it feels like something new comes out of AI every day. I think every creative lead, at an agency or in-house, is feeling that. There's just not much time to test everything, and if you scroll Instagram it looks like you're falling behind. Most likely you're not.

There's a new model every month, and each one is better at something different. The hard part is knowing which one to use for which type of work.

‍

Everyone testing everything

For a while at Pearmill, all of us were testing. Everybody had a project, their own skills, their own folders. Someone would try a new tool, get excited, and put it in Slack. Maybe a few people would catch up on it, but it was tough for anyone to keep being exposed to new things and actually adapt to them.

‍

What we do instead

Part of my role right now is proof of concept. I really like just trying things out. If I have a task, I'll run it in three different tools and see which one does it better. That curiosity is what made it clear to the rest of the company that I can try things and share them with the team every week.

‍

This way the team works in setups we've already created. Nobody has to feel guilty about not testing something that week, because someone on the team already is.

I know there's probably an AI tool that could do this testing for me.

‍

I still prefer to do it myself.

‍

Same model, different results

Something I didn't expect: I don't know what these companies are doing around capping capacity for different models, but results move. Two weeks ago I tried ChatGPT, was impressed, and put my card down for a bigger plan right away. Two weeks later, the same model was giving me subpar results on the same type of tasks.

‍

I also think a lot of the mental block is trust.

‍

People doubt that the AI can do a good job. And most of the time, if you don't set the model to extra effort and use the newest version, you won't get anything good out of it.

Where I am right now

I'm starting to see patterns. We're using AI a lot for motion design at the moment. I've tried ChatGPT, Claude, and Claude Design, and Claude Sonnet 5.5 is fantastic for motion design, including work inside After Effects if you need it.

‍

For UI and landing page design it's different. A few weeks ago I only used ChatGPT. Now I'm using Claude more, because ChatGPT has been giving me really bad results.

‍

The lesson is that you can't stop testing, because even the same model will probably give you different results in a few weeks. That's why I get why people are tired of this.

‍

Skills are changing too

Skills used to be about how to use a tool. Before, I might have had a skill for how to create something simple in Figma. Now I can just ask for something simple in Figma and it does it right away, no skill needed.

So for me skills are moving toward refinement. I give feedback until I reach a final result, then I package that up as a memory or a skill so the next time starts from everything we learned.

‍

Trust

I was talking about this earlier, but a lot of people still don't trust these tools, and each person has a different level they're comfortable with. Personally, I give it auto approvals all the way through. I've very rarely had issues. One time I got things overwritten and couldn't get them back. Other than that, nothing.

‍

But we did have people give it their Figma files, and it ended up hiding things or moving things that shouldn't have moved. So most of the time I give it its own Figma board and tell it not to go anywhere else. I prepare everything it needs in that one board and let it work there.

If you take one thing from this

Find one person at your company who's curious about AI tools. Give them a task every week: bring back what they learned to the team. It helps everybody.

‍