After its launch in November 2022, ChatGPT changed the dynamic of how mainstream consumers obtain information. Though still not perfect, it brought an improvement that trickles down into advanced customer service applications, i.e., chatbots. However, the use of large language models (LLMs) in general goes beyond just dealing with customer complaints.
Large language models have been applied to cybersecurity and have brought even better contextualization for threat detection and response. LLMs are developing rapidly, and innovations in the near and far future will push the generative AI market to grow at a CAGR of 35.6% between 2023 and 2028.
As with any technology, the use of LLMs brings certain risks to the integrity of cybersecurity systems.
Our article dives deep into everything you need to know about the impact of LLMs on your organization’s cybersecurity.
Also Read: Machine Learning and Artificial Intelligence in Cybersecurity
What is a Large Language Model?
A large language model (LLM) is a form of deep learning artificial intelligence (AI) trained for natural language processing (NLP).
It’s basically an algorithm that is fed from a “large”/massive set of l data, through which it learns the appropriate language syntax.
The large language model, through its understanding, is then able to interpret, analyze, and generate synthetic human-like sentences/textual information.
LLMs are primarily trained to predict the next word (or token) in a sequence. They work through a parameterized statistical model that defines just how the model behaves and helps with determining probabilities and context.
The key parameters/variables that determine how an LLM performs include:
- The parameter model size, which relates to how many parameters are used in the LLM analysis.
- The token size, which is the number of words and abstract characters the model is trained on
- The determined context window, which is the number of tokens considered when creating a vector for analysis and prediction.
- The set temperature, which determines the LLM’s freedom and creativity in its use of words
- The top-k and top-p after analysis. Top-k considers the “k” number of tokens with the highest statistical probability to be next in the sequence. Top-p, on the other hand, uses a tight cut-off probability score to grade tokens for consideration.
Comparing GPT-3 and GPT-4 shows us how the parameter model size is core to the function and depth of any LLM algorithm. While GPT-3 uses 175 billion parameters, GPT-4 is trained on over 1.76 trillion parameters, hence why it is the more advanced LLM.
What’s amazing about LLMs is that the use of billions of parameters allows them to go beyond natural human language generation. We see them with the capabilities of interpreting and generating software code.
The combination of LLMs’ human and computer language processing functions then presents important cybersecurity use cases for organizations.
How organizations are applying LLMs in cybersecurity
Whether coming as original applications of AI or improvements over existing AI-powered processes, LLMs offer cybersecurity professionals a vast range of new ways to protect data and infrastructure.
The most common of these include applications in:
1. Security posture management
Large language models can be used to hasten security posture risk assessment procedures.
Security posture management is an important preventative cybersecurity measure that involves identifying and eliminating configuration and compliance risks from the IT environment.
LLMs can be trained using deep datasets that define both optimal and risk-exposed IT infrastructure states. They are then able to dig into log files and procedural documentation. The goal is to identify instances of misconfigurations, existing vulnerabilities, or non-compliance with NIST, SOC 2, and any other standard they are trained on.
Also Read: Differences Between SOC 2 and SOC 1 Reports
Through an integrated AI interface, your organization’s IT team may extract information on where vulnerabilities exist or what devices are exposed to compliance risks.
The LLM can also help to carefully list out the best steps to eliminate current cybersecurity risks. It does this based on its training on optimal IT states and its analysis of current risk-exposed states.
2. Software debugging
Remember that, due to the use of statistical models, LLMs are not just able to interpret or generate only human language. They may also be trained to process computer language or any other means of communication between two entities.
This means they can analyze software code and identify where syntax errors exist or whether there are loopholes in scripts and code logic.
The best thing about LLMs here, however, is that they can combine natural (human) language processing with computer language generation and vice versa. A natural language prompt can be used to direct an LLM into a code repository.
The LLM identifies bugs in the code and, based on data it was trained on, can create documentation explaining the steps toward fixing the problem. It may even present the exact code that solves the problem.
The GitHub Copilot serves as an excellent example of LLMs for software debugging purposes.
3. Threat Detection and Response
The area of threat detection and remediation is where LLMs provide the most value to organizations.
Here, LLMs can be used for threat hunting, where the algorithm is continuously trained on new pieces of information about threats and vulnerabilities.
Rather than being trained on a static dataset, an LLM can be provided access to new pieces of information brought to the internet via forums, social media posts, blog articles, and research paper releases.
The LLM may also be trained on information generated after every internal security assessment or other types of documentation.
Through this form of training, the LLM understands new forms of attacks even before the human cybersecurity team. It immediately puts these into context when parsing through logs for weaknesses and signs of breaches, increasing the accuracy of vulnerability scans.
Thanks to NLP capabilities, an LLM can also be used to
- Facilitate text-based behavioral analytics, helping to spot a change of personnel behind the keyboard.
- Identify phishing content in emails and IT conversations after being trained on what successful phishing looks like.
- Spot malicious prompt injection attempts to protect external-facing chatbots and bolster internal-facing AI assistants against equally-evolved threat actors
LLMs can also be used to facilitate remediation.
They can analyze alerts or data deposited into system logs, understand the full context around the attack perpetuated, and offer the best steps toward remediation for the IT team to follow.
This application is especially useful within complex IT environments where security analysts may spend a costly amount of time determining remediation steps.
What’s more, your security team can save even more time when LLMs help to generate cybersecurity reports.
Also Read: Email Attacks and How to Protect Your Organization Against Them
Case study of LLMs in Cybersecurity: Sec-PaLM 2
We can use Google’s Security AI Workbench to have a good picture of the application of LLMs in cybersecurity .
The Security AI Workbench is powered by the Sec-PaLM 2, a Large Language Model specifically designed for cybersecurity applications.
The solution, used by cybersecurity-focused Fortune 500 company Accenture, helps to achieve the following:
- Analysis of IT data
- Generation of security designs
- Generation of configurations and controls
- Spot malicious code
- Summarizing threat data
- Prioritizing events.
The Security AI Workbench also allows security personnel to search for events using natural language. It also helps to interpret complex attack graphs into plain textual language.
How will Large Language Models impact cybersecurity
The use of LLMs is regarded as a double-edged sword. This means that its impact has a positive side and a negative side. We look at these two sides from a benefit perspective as well as from a risk perspective.
Beneficial impact of LLMs on cybersecurity
From the different applications of LLMs, we may then pick out certain benefits they bring to the cybersecurity landscape.
LLMs can:
- Eliminate the need for manual data analysis
- Enable the analysis of unstructured data
- Help to create incident response scripts in an instant
- Identify all sorts of abnormal behavior
- Hasten contextual information retrieval for security response.
What’s more, LLMs come with emergent abilities.
Emergent abilities are new, unexpected capabilities that spring up as the training data grows larger. The more parameters the model is trained on, the more creative it becomes in solving cybersecurity issues.
Hence, when trained deeply enough, LLMs eliminate complexity in an organization’s cybersecurity operations.
Risks of LLMs to cybersecurity
The major risks accompanying LLMs in cybersecurity come as exploitations on both the client side and server side of an LLM application.
Although there are multiple risks, one proves to be more integrated into LLM functionality, more so that it can be utilized on both the client and server sides. This is the prompt injection attack, which we have discussed at length. Please check this complete guide into chatbot prompt injection attack as an emerging threat.
Other risks of LLMs to cybersecurity include the creation of Convincing Phishing messages, Dictionary Brute-force attacks, Malicious Code generation, and Training Data Poisoning.
1. Creating convincing phishing messages
LLMs can be used to create messages in different tones and with multiple levels of creativity. Alongside their ordinary ability to generate creative sentences, cyber attackers may also train them using data scraped from the target’ individual’s social media accounts.
Using this as context, an LLM can assist cyber attackers in generating very creative, personalized phishing messages for any form of campaign. Employees can easily fall victim to this.
2. Dictionary brute-force attacks
A dictionary brute force attack utilizes a list or «dictionary» of common password combinations to support password-guessing operations.
An LLM can assist in scraping conversations between IT employees or data in log files. It can be trained on these pieces of information and then used to generate a brute-force dictionary that is personalized specifically to target your organization.
3. Malicious code generation
Cyber attackers can use an LLM to trick or outrightly use them to generate malicious code. This code can then be used to build a malware function or execute a malicious operation in the internal components of IT infrastructure linked to it.
4. Poisoning of training data
The corruption of LLM training databases with false information is a risk organizations should be particularly wary of.
An LLM may be poisoned by an attacker to ignore certain prompts or vulnerabilities, or trained to execute certain tasks only the attacker knows how to exploit.
This is even more worrisome when you consider that, with the use of methods like the TrojanNet backdoor technique, an attacker doesn’t even need access to the original training database. A simple trojan module may be injected into the LLM and the malicious operation can be triggered by a special input token.
One other risk posed by LLMs is Hallucination. Here, LLMs may present factually wrong outputs to an organization’s IT response team. This then creates false positives or inactionable recommendations for remediating incidents.
How to mitigate the impact of LLM-related risks?
To limit the risks LLMs pose to cybersecurity, it is crucial that organizations take certain actions during model training and data retrieval.
Some of these measures include:
1. Adversarial training
Adversarial training involves teaching LLMs how to recognize threats by providing them with examples of cyber attack instances.
The cyber attack instances can be either a common prompt used for malicious input injection or highly successful phishing messages.
Once the LLM understands related malicious tokens and syntax, it can execute a more effective stop sequence and even send out alerts about malicious activity.
2. Defensive distillation
Defensive distillation is a more advanced type of adversarial training that involves two machine learning algorithms.
The first «teacher» model has access to raw training data, while the second «student» model is trained on only the outputs of the first teacher model.
Through predicted patterns, the student model creates expectations for outputs from the teacher model. It easily identifies instances of spoofing or injections from malicious actors when its «teacher» outputs don’t fit its predictions.
3. Federated learning
Federated learning is a decentralized method of training large language models. Instead of giving the algorithm access to a major depository of raw data, this method involves training smaller models with local data and depositing separate information into a large database. The LLM is then trained using information supplied to this database.
4. Gradient masking
One method used by attackers against machine learning models like LLMs is the Fast Gradient Sign Method (FGSM). It involves computing the loss or variation between expectations and a model’s output.
The attacker then uses the loss to compute a gradient, which is carefully exploited alongside an output result to increase the model’s loss. This makes the model misclassify inputs or produce wrong results on similar inputs.
Gradient masking involves adding regularizing elements that modify the gradient each time it is computed. This way, the attacker finds it more difficult to generate an exploitable (non-zero) gradient.
These additional approaches are also effective in protecting against LLM exploitation:
- Encryption of training data
- Anonymizing data to remove personal identification
Also Read:
Conclusion
The LLM and generative AI market is expected to gain over $40 billion within the next seven years.
However, there is still one major downside of LLMs that threatens even the likes of GPT-4. This is the use of static datasets to train the LLM algorithm. Static datasets are a barrier to active threat hunting, as they prevent the LLM from adapting alongside ever-changing threat strategies.
As mentioned in a Forbes publication, the way is to adopt a new form of foundational training technique. This is a technique that permits quick, dynamic, continuous, and spontaneous LLM learning.
It is by doing this that organizations can gain an upper hand in the arms race against cyber-threat actors.
