Rendered at 07:57:41 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
simonw 2 hours ago [-]
On the one hand this is exactly the right solution to prevent lethal trifecta exfiltration attacks.
The existence of lockdown mode does however imply that ChatGPT, in its default settings, does not provide robust protection against sufficiently determined data exfiltration attacks!
Yeah I know the source references him (replying to his comment), that's exactly why I'm giving credit where it's due
Noumenon72 1 hours ago [-]
I hadn't realized that deep research or generating images that I paste into Twitter were possibly exfiltrating my data. Yikes.
kirtivr 36 minutes ago [-]
Is this an admission that prompt injection attacks can indeed not be blocked by an analysis based technique?
If so many tools are straight up blocked, I would be very sceptical of the quality of the results.
sigmoid10 32 minutes ago [-]
I think "prompt injection prevention" systems fall into the same category as "llm writing detection" systems. I.e. reality is always a step ahead and you shouldn't trust either one for anything remotely important.
There is something so off about him for me that he makes my skin crawl.
Always has been before he was associated with OpenAI.
Which is weird because the bullshit he spouts isn’t so different to the bullshit other top execs spout and I don’t have the same visceral reaction to them (though I still don’t like a bunch of them).
I imagine that enterprise companies will be quite interested in this.
zerobees 2 hours ago [-]
"Prompt injection is not currently a major risk, but its impact could grow as attackers develop more sophisticated methods." - that's such a weird statement to make. It's one of the most significant factors limiting the adoption of the technology in business.
I have mixed feelings about this feature. We're playing with tech that's supposed to do human-shaped things but can't be trusted nearly as much as a human employee (and can't be held responsible for what it does). Restricting the tools available to that patently untrustworthy entity doesn't solve the problem, it just makes the entity less useful, forcing you to sooner or later let it out of the jail.
noir_lord 6 minutes ago [-]
> forcing you to sooner or later let it out of the jail
Suspect thats the point, by giving you the “choice” they also make the user responsible or can at least shift the blame.
ACCount37 42 minutes ago [-]
Responsibility is worthless for humans and even more worthless for AIs. In a way, AIs just make it more obvious.
And "trusted nearly as much as a human employee", well... you do know that phishing and insiders are two primary ways for attackers to get into company infrastructure, right?
AIs pair human-shaped capabilities with human-shaped vulnerabilities. It's a way of automating PEBKAC.
kijin 4 hours ago [-]
So we still don't have a reliable way to separate instructions from data when talking to an LLM, a problem that humans learned how to solve decades ago in areas like SQL and memory safety. But hey, we have these hopefully-not-leaky containers, which are probably implemented with just more system prompts.
How long until somebody figures out how to trick Codex into disabling Lockdown Mode for you?
mapontosevenths 2 hours ago [-]
> So we still don't have a reliable way to separate instructions from data when talking to an LLM
Humans also do not know how to do this reliably, which is why phishing is still a thing and always will be.
Smaug123 2 hours ago [-]
I think the Stroop effect ("read these colour names, each written in a different colour") is probably the purest demonstration of this. Humans are trivially prompt-injectable.
dnnddidiej 4 hours ago [-]
We can seperate them but the $ value of an agent that does is much lower than one that doesn't.
As a pre LLM analogy imagine working at a bank with a whitelist firewall. You need to install a package but requires an IT ticket. Safer but slooooower.
Now not saying what the answer here is but that is the issue.
The answer may be more like industries that get safer through lessons (like aviation) rather than go for 100% safety out of the gate. Because both fast travel and AI agents are insanely useful.
altmanaltman 3 hours ago [-]
what? Aviation safety is not designed to get safer through lessons? They literally try to ensure it is 100% safe out of the gate. The accidents that happen are usually statistical outliers and lead to loss of life.
That's what it means when they say aviation regulations are written in blood. Not that they just fling planes into the sky and be like "boy i hope we learn some new regulations from this". The number of airplane crashes would be astronomically larger if the 100% safety part was not embedded into the design process.
dnnddidiej 3 hours ago [-]
I think we agree? Unless my reading comp is off today.
4 hours ago [-]
madanparas 4 hours ago [-]
The help doc explicitly carves out Codex: "Lockdown Mode does not affect network access in Codex." The mode limits outbound requests in chat to block prompt injection exfiltration, but Codex network access is a separate setting. An enterprise team that turns on Lockdown Mode while using Codex against internal repos still has an open outbound path this mode doesn't cover.
The existence of lockdown mode does however imply that ChatGPT, in its default settings, does not provide robust protection against sufficiently determined data exfiltration attacks!
If so many tools are straight up blocked, I would be very sceptical of the quality of the results.
Always has been before he was associated with OpenAI.
Which is weird because the bullshit he spouts isn’t so different to the bullshit other top execs spout and I don’t have the same visceral reaction to them (though I still don’t like a bunch of them).
I imagine that enterprise companies will be quite interested in this.
I have mixed feelings about this feature. We're playing with tech that's supposed to do human-shaped things but can't be trusted nearly as much as a human employee (and can't be held responsible for what it does). Restricting the tools available to that patently untrustworthy entity doesn't solve the problem, it just makes the entity less useful, forcing you to sooner or later let it out of the jail.
Suspect thats the point, by giving you the “choice” they also make the user responsible or can at least shift the blame.
And "trusted nearly as much as a human employee", well... you do know that phishing and insiders are two primary ways for attackers to get into company infrastructure, right?
AIs pair human-shaped capabilities with human-shaped vulnerabilities. It's a way of automating PEBKAC.
How long until somebody figures out how to trick Codex into disabling Lockdown Mode for you?
Humans also do not know how to do this reliably, which is why phishing is still a thing and always will be.
As a pre LLM analogy imagine working at a bank with a whitelist firewall. You need to install a package but requires an IT ticket. Safer but slooooower.
Now not saying what the answer here is but that is the issue.
The answer may be more like industries that get safer through lessons (like aviation) rather than go for 100% safety out of the gate. Because both fast travel and AI agents are insanely useful.
That's what it means when they say aviation regulations are written in blood. Not that they just fling planes into the sky and be like "boy i hope we learn some new regulations from this". The number of airplane crashes would be astronomically larger if the 100% safety part was not embedded into the design process.