Skip to main content

Symflower for RAG & enhancing code generation for LLMs

Symflower can be used as an effective tool to improve the accuracy of existing generative AI (large language) models for code. Through Retrieval Augmented Generation (RAG) for LLMs, you can leverage Symflower's capabilities to:

  • Augment prompt engineering: Query analytics tools to get more context about source code. Compute values to reach certain areas of source code. In our experiments, augmenting ChatGPT with a Symflower test template showed better results than using ChatGPT alone.
  • Provide higher quality training data: Curate training data through using Symflower to filter projects with problematic code.
  • Enhance the accuracy of LLM models for code: Use Symflower to fine-tune code generation by providing Symflower's generated tests as training data to LLMs.
  • Evaluate the quality of code generated by LLMs: Our DevQualityEval establishes benchmarks (code coverage, number of found bugs, etc) for measuring the quality of generated code.

Features for RAG​

Running symbolic execution on a main function​

Symflower's CLI version lets you send a main function to the symbolic execution engine and returns coverage as a result. This enables you to run your Go or Java programs using Symflower's symbolic execution. Here's how to use this feature:

symflower run $path-to-a-file-with-a-main-function

Here's an example of the output you'll get:

Execute your Go or Java programs using Symflower's symbolic execution."

Query all functions/methods in a repository​

If you're trying to get an LLM to analyze all functions/methods in a repository, you'll need to make sure all of the files are located, all the function/method names are found, and maybe even that the locations of functions/methods within the file are pinpointed. Since Symflower statically analyzes your code to generate tests, it already has all that information.

Use the following command to access a list of all the symbols (methods/functions) in a given repository (this is a language-independent feature that works for both Go and Java):

symflower symbols

Here's an example of what a log of all the symbols in a file may look like:

Use Symflower's static analysis to find all functions/methods in a file."