bleepingcomputer[.]com/news/security/time-bandit-chatgpt-jailbreak-bypasses-safeguards-on-sensitive-topics/
In a nutshell, big money AI like OpenAI trained their models by feeding them the internet. Because of that when you ask ChatGPT a question it might give you an answer that might take you hours to find, or maybe you wouldn't have been able to find it at all. One of the big concerns about safety is that there are some things that really shouldn't be easy to find, like making bombs, malware etc. So companies like Meta, Microsoft, OpenAI etc. all try to build in safeguards so their AI won't tell you something you probably shouldn't know. But AI is software, so it can be hacked, and asking the right questions can jailbreak the model so it tells you what you want to know. A security researcher found a way to do that, but unlike finding a problem with a popular app, he couldn't find anyone at OpenAI that was willing to listen. The same thing happened with CISA, the FBI, & other US gov agencies. He contacted Bleeping Computer, and they couldn't get a response out of OpenAI either. Contact was finally made via the CERT Coordination Center's VINCE vulnerability reporting platform, though the problem's only partially been fixed.
There are likely many thousands of people trying to figure out how to jailbreak ChatGPT & other AI models, and the odds are that some of them are successful, which you would hope would be very concerning to a company like OpenAI -- something that they're looking for and trying aggressively to stop. And it seems that you would be wrong.