Stealing Part Of A Production Language Model

In this whitepaper, the authors introduce the first model-stealing attack that extracts precise, nontrivial information from black-box production language models like OpenAI's ChatGPT or Google's PaLM-2. Specifically, their attack recovers the embedding projection layer (up to symmetries) of a transformer model, given typical API access. For under $20 USD, their attack extracts the entire projection matrix of OpenAI's ada and babbage language models. They thereby confirm, for the first time, that these black-box models have a hidden dimension of 1024 …

access api attack authors box chatgpt google information language language models openai palm production stealing under whitepaper