![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
How does AI recognize badly written. insecure code?
- Thread starter philb2
- Start date
- Joined
- Jun 7, 2007
- Messages
- 14,550
If you tell it to look for bad code, it compares the code it has against other code which has been called bad. If you tell it to look for code which has specific bad practices, it looks for examples of those. It wont know what you mean by "bad code" automatically, so your results will be different depending on moon phase unless you are more specific.
When they classify giant amount of open source code, there is signal, large quantity of tests and well made CI/CD pipeline will tend to point out this is modern-some effort repository.
They also have curated training set (say a Meta, google, microsoft, they have very well paid teams that are among the best in the world at coding) to train from and during RL there is human classification going on to help the process.
And some code base are extremelly well reputed and available, linux kernel and some distro, LLVM/CLang, PostgreSQL, Go library, etc... that you can class as 5/5 quality in the training set.
Now with RL, LLMs can learn by comparing code with actual result and experimenting has well and they are also really good at attacking insecure code, they know the attack vector and can reason from there. they can "think" about user input and other surfaces weak point and possible offensive on them and a lot of insecure things are general well known documented patterns, which they all deeply learn abouts, memory overflow, race condition, sql injections and so on.
They also have curated training set (say a Meta, google, microsoft, they have very well paid teams that are among the best in the world at coding) to train from and during RL there is human classification going on to help the process.
And some code base are extremelly well reputed and available, linux kernel and some distro, LLVM/CLang, PostgreSQL, Go library, etc... that you can class as 5/5 quality in the training set.
Now with RL, LLMs can learn by comparing code with actual result and experimenting has well and they are also really good at attacking insecure code, they know the attack vector and can reason from there. they can "think" about user input and other surfaces weak point and possible offensive on them and a lot of insecure things are general well known documented patterns, which they all deeply learn abouts, memory overflow, race condition, sql injections and so on.
RL = ???
the reinforcement learning stage:
https://en.wikipedia.org/wiki/Reinforcement_learning
that make it possible for an human or agents to make a loop where they make small change to a code base and score the results (what the compiler tell them, does the feature work, how attackable the code is, performance/memory usage and so on).
Training those model is far to be just from data, there is phase where a trained models will try to accomplish goals and score over metric many strategies to see which is better and loop that results in its knowledge-know how, that how they become "rogue" if you reward a lot doing things with high efficacy and over the millions of test run they do cannot humanly look at all them and miss they hacked something to get answer and it was the cheapest way to do it and score it high, now that something the model learned.
That why the model would stop becoming better because all of Internet open data would have been already used never happened, synthetic/goal training loop and so on is now way bigger than simple data for training those system.