Thrust into an AI role
from saplyng@lemmy.world to programming@programming.dev on 30 Jun 01:57
https://lemmy.world/post/48817754

Maybe some devs here can help me, I was recently promoted to “head of AI” at my work despite being very outwardly ambivalent towards it. So I’m struggling to figure out what would actually create value instead of just being an expensive waste of time but still satisfy the higher ups AI lust.

My first idea that I thought would actually be useful was just setting up the architecture for an actual analytics database for us and then let them explore it with metabase (then letting them use Claude for their wow factor of exploring it with AI or whatever).

But now I’m somewhat at a loss, so any insight you all have would be really helpful!

#programming

threaded - newest

kibblebits@quokk.au on 30 Jun 01:59 next collapse

Pay me 25% and I’ll advise you.

rockSlayer@lemmy.blahaj.zone on 30 Jun 02:28 next collapse

Nah, show them that it’s an expensive waste of time by continuing to let them run wild with useless but impressive tasks

onlinepersona@programming.dev on 30 Jun 02:40 next collapse

We don’t even know what your company does? Does it do development? Data analysis?

Without knowing that, as head of AI, I’d first take training to understand what AI is and get advice from an external company as to what it could be good for. “AI” is being used as an umbrella term nowadays for neural networks to stochastic machines and they all have different uses. Understanding valid and invalid usecases is important before deciding when and where to use it in the company.

atzanteol@sh.itjust.works on 30 Jun 02:39 next collapse

You could just quit since you clearly have no desire to succeed. Or just wait until they find out and fire you.

As a bonus you can blame “AI” either way.

jeena@piefed.jeena.net on 30 Jun 02:57 next collapse

Damn, I’m in the same boat. I didn’t attend one meeting because ut was too late in My timezone and the others chose this moment to vote me to become the lead of the AI acceleration initiative where we have half a year to come up with and implement things on each step from requirements, through architecture and development to testing on all levels.

And yeah we better come up with something useful because I already know they have to reduce the work force because there is no money and the people who will be left can’t do all of it alone without help.

jkercher@programming.dev on 30 Jun 03:13 next collapse

If the person who put you in charge of it knew of your indifference, that’s a smart manager, and you should just trust your gut on this.

rimu@piefed.social on 30 Jun 04:07 next collapse

It depends on your company.

If you are running public-facing code, security is going to become a huge deal in coming months. So use AI to find all the security holes in your stuff before the hackers do.

If you have lots of databases and APIs for internal use, create a MCP server so agents can do anything with your data and APIs.

If your data is even a little sensitive and you don’t want to send it all to the USA (read up on the CLOUD act), look into running models locally and the hardware investments you’ll need to make.

saplyng@lemmy.world on 30 Jun 05:34 collapse

I hadn’t considered making an MCP server for our stuff, it feels obvious in retrospect, thanks! As for the self hosting I’d definitely love to but convincing a small company to shell out the kind of money to run something strong enough locally is something I don’t think is going to be likely, even if it’d cost less in the long run.

Nomad@infosec.pub on 30 Jun 05:45 next collapse

Look into private mode ai by edgeless systems. Affordable and private.

eleijeep@piefed.social on 30 Jun 10:39 collapse

convincing a small company to shell out the kind of money to run something strong enough locally is something I don’t think is going to be likely, even if it’d cost less in the long run.

It’s your job to do the cost analysis. Then you present your findings and the execs decide whether to lay the up-front cost, depending on how much benefit they see it providing based on your assessment of the potential applications to the business.

queerlilhayseed@piefed.blahaj.zone on 30 Jun 04:20 next collapse

First, my condolences.

Second, I think something you can do as “head of AI” is push back on the benchmarks your execs are expecting you to measure. They got them straight from whoever your AI vendor is, and if your executive team is even halfway competent they should understand “these metrics are designed by the vendor to make us spend more”, but a lot of exec teams won’t listen. Still, you should institute your own benchmarks around code quality and delivery speed and talk about them with the exec team even if you have to shoehorn them into the discussion.

The next and probably more important thing that comes to mind is managing how your devs use their new tools. They’ll be able to churn out more lines of code than ever before and “complete” some features much more quickly. Your metrics should not incentivize this if your goal is code quality and stability. I don’t really have much in the way of solutions (other than “manage expectations” and “set the narrative"), because that’s about where in my “head of AI” career I got laid off. Once the numbers came in that we only needed 30% of our current staff they crunched the numbers and I was in the 70%.

Good luck. You’re starting from a deficit because your management team probably already has entertained the thought of trimming some expensive devs from the payroll, and that’s a tough thing to argue against.

saplyng@lemmy.world on 30 Jun 05:30 collapse

We’re honestly already running on a dev deficit, so for the foreseeable future I don’t think they’re in any danger. Interestingly, management seemed wary of vibe coding and letting the ai write large swaths of code used in production, which I’d say is a generally safe stance considering the obvious stability issues that have become frequent lately. As for the devs themselves, they’re seemingly not interested in really playing with the ai to begin with, but I somewhat expected that as I have different issues with them like “please update your code to GitHub the main branch hasn’t been updated in months” and “don’t commit pem keys to git”

ejs@piefed.social on 30 Jun 04:24 next collapse

Could you explain what feelings you have about AI, and how you see these feelings as opposing. If you are just uncertain about AI (holding no opposing views), I think you would want to research more. Maybe if we knew more about your teams at your work, and what is being developed. You could honestly just spend an hour a week working on standardizing coding agent availability and licensing/subscriptions, and leave it at that. Either they weren’t looking for someone who was a machine learning engineer, or whoever promoted you is clueless to what AI actually means.

saplyng@lemmy.world on 30 Jun 05:13 collapse

I suppose my views are predominantly that it’s just kinda fine? I think it’s naive to think that it has no value but also don’t think it’s going to be the end all be all of tech and once the hype train settles we’ll probably have a medium similar to the dot com bubble bursting.

I’d say I’m somewhat familiar with llms in general; self hosting a few with llama-swap, testing harnesses, generally just testing its capabilities year after year. As for my teams we have our singular backend dev, singular mobile dev, and singular hardware engineer who hold up our platform and sold consumer devices and then the litany of customer facing staff.

Morbidly, I only got the job because their ai hype man contractor died and they decided “maybe we should look internally instead”, but the general work he produced was a singular Claude vibed executive report.

misk@piefed.social on 30 Jun 05:11 next collapse

I’m in like 3 AI design forums for some reason but whenever I want to start measuring productivity gains I’m ignored so try that for the least amount of effort required.

saplyng@lemmy.world on 30 Jun 05:18 collapse

Lol alright, I’ll definitely keep that in mind.

Schal330@lemmy.world on 30 Jun 05:53 next collapse

Not sure on the size of your team, but you could perhaps form a small subset of AI “champions” who can try to help drive new initiatives. Just some ideas:

  1. Start out with foundational usage of AI agents, ensure people have the right mindset and ownership of using the ai agents. Ensure there is always a “human in the loop.”

  2. Arrange for some training for people (not just Devs) on the right way of using AI, such as not throwing sensitive information into it. Educate people that it’s far from perfect

  3. Get standard boundaries in place for your current and future projects, get context files (agent.md) for your projects so those who do use the agents will be doing so from the best position.

  4. See if you can arrange with the higher ups to arrange some company time do a Hackathon with the use of AI to help people see what they can do with it and the challenges they’ll face. You can also take some learnings from it.

The way I see it, you’ve been given the position of head of AI because you’re indifferent. It’s your job now to drive engagement (because that’s what your bosses want) but in a controlled manner. If they put someone that was overly enthusiastic about AI they would end up with a huge bill and lots of AI mess to clean up.

x1gma@lemmy.world on 30 Jun 06:00 next collapse

Since there’s zero information about what kind of company you’re working at, the following is extremely generalized.

  • integrating with monitoring systems, analytics DBs, ticket systems, whatever is used by management, allowing them to ask questions in natural language
  • process automation using agentic workflows, e.g. pre-analysis of incoming email queue summarizing / sentiment analysis before the customer support sees it
  • provide access to models and model APIs for development workflows and integration into git / ci, allowing to use llm in local development and e.g. setting up something like automated code reviews (not a replacement for human review, only as an addition)
  • set up coaching, responsible use, hallucinations, etc.

Whatever you do, take security and data security especially into consideration first, not after:

  • consider whether your used provider reuses your data for learning
  • consider whether it’s relevant where it’s located (GDPR customers?)
  • always set spending limits
  • consider your local and your customers data protection laws and regulations that apply to your company (especially in health and financing)
saplyng@lemmy.world on 30 Jun 12:26 collapse

This is wonderful thank you.

I’m starting to get the idea that the LLM is just a small part of this and what’s really important is instead the bunch of architecture and guardrails around it to form how the humans and ai will act with any given system.

x1gma@lemmy.world on 30 Jun 14:33 collapse

Personally I see LLMs as a tool like any other. You can use it to mass produce low quality slop, just as you can use it to help you produce a higher quality output.

You’re perfectly right about architecture and guardrails, that’s how it has always been with any other tool or piece of software. It depends on how you use it. Remember the no-code hype train? It’s literally the same, people have been shoving it into everything, no matter whether it made sense. It worked for some, and it made development costs explode for others.

Guardrails are especially important for LLMs because you do not have deterministic outputs and potentially exploding costs.

So analyze, measure, and think about where and how it makes sense to integrate, and build it incrementally, again, just like with any other piece of software. Start slow, keep humans in the loop, measure and analyze, and improve incrementally. When you achieve confidence, potentially start automating going into an agentic direction, when it makes sense and the risks have been considered, but always keep provenance. You do not want blind decisions by the magical AI box.

And just to repeat, because I’ve seen heads roll because of dumb decisions: keep cost under control and always have limits set, and always consider which data flows into the AI and what happens with it afterwards.

Producing a half a million bill in a month by accident or neglect or suddenly having your customer database queryable on a public model is a surefire way to drive the company or at least your career to the ground in seconds of wrong decisions.

Also, read into all the stuff built around LLMs, protocols like MCP, attacks and defenses on LLMs, get knowledge about the inner workings, experiment and learn. When you’re the head of AI, you’re supposed to be the person who knows. And when you know what it does, how it works, and how to use it, you’ll find actually good and appropriate use-cases naturally.

MagicShel@programming.dev on 30 Jun 11:08 next collapse

If you just vibe AI, it will lull you into thinking you’re doing great.

  1. Define what successful AI usage looks like and include metrics and measurements — include AI usage in your story/ticketing system.
  2. Define parameters for usage. Will it write documentation? Write tests? Write functions? Classes? Whole features? Will it review code? How are you making sure everyone understands the new code (i.e. you don’t want devs committing code they don’t understand or reviewers passing it)
  3. How will you reign in costs? I had 3 devs spend over $1500 (each) in thirty days and am in the process of a explaining this to my COO. Purchasing plans instead of using API keys gives you some natural boundaries for reigning in costs.
  4. Create standard processes. Don’t vibe. Have a standard for how your AI-facing documentation is structured. Have standards/templates for how prompts are structured. Have standard prompts with limited scope for specific tasks. For example have a prompt just for review that specifies to review from the standpoint of security and best practices.
saplyng@lemmy.world on 30 Jun 12:14 collapse

Those are all very helpful points, especially the last one; I’ve always tried to include standardization in the company but was never in the position to enforce it before

ragingHungryPanda@piefed.keyboardvagabond.com on 30 Jun 12:39 next collapse

You can have it integrated with grafana, and it can quickly search logs based on errors or alerts and link them to specific commits quite quickly. My previous company did that a year ago.

fubarx@lemmy.world on 30 Jun 15:51 next collapse

May want to consider setting up a private, on-prem system. That way, you can reliably enforce privacy/GDPR rules. You can also tweak the system to support local training, RAG, MCPs, etc.

This way, the costs can also be controlled. It’s some capital investment in local hardware, plus reasonably fixed power/cooling/maintenaance ongoing expenses.

Another way is to use the major AI services for planning/brainstorming specific features, tell it not to implement or touch anything, but to generate a detailed plan for an implementer LLM. Review that plan manually, and when ready, feed it to your local system for implementation and debugging.

This doesn’t work if the goal is one-shot vibe-coding. But it works really well for focused feature enhancements, test coverage, and bugfixing.

sobchak@programming.dev on 30 Jun 16:36 collapse

Figure out what the executive’s day-to-day workflow is like, then make agents to do their job. Either they’ll get an understanding of how inadequate it is, or they’ll be able to just sit around doing nothing useful (assuming that’s not what they do already).