Skip to main content

FAQ

What are the main features of Symflower?​

Symflower's features cover testing and LLM-related functionality.

For software testing, Symflower extends your development environment with 3 new features:

For working with LLMs, Symflower offers the following features:

  • symflower fix: a static analysis tool to repair code generated by LLMs
  • symflower lint: runs static analysis with preconfigured linting rules on your project
  • symflower migrate: migrates code, packages, dependencies and whole projects automatically
  • symflower run: sends a main function to Symflower for symbolic execution and returns coverage as a result
  • symflower symbols: queries all functions and methods in a repository and outputs a list of all the symbols
  • symflower test: executes the tests in your project and outputs test results and coverage information
  • symflower test-runner: performs package-level test impact analysis on your Go repositories to identify tests that are affected by the source code changes specified in your query
  • symflower trace: a task visualization tool that provides insights into application execution behavior.

In addition, the Symflower team also develops DevQualityEval, an evaluation benchmark and framework to compare and evolve the quality of AI software development agents. See the latest deep dive for details, and support DevQualityEval to access the leaderboard summary, graphs, and exported metrics.

How do I install Symflower?​

For a walkthrough of the installation steps in different environments, see the Installation section of this documentation.

How do I get started with Symflower?​

For help in taking the first steps with Symflower, see the Getting Started section of this documentation.

Where do I find more resources and examples to help get started with Symflower?​

This documentation offers a tutorial for smart test template generation for (plain Java and Spring Boot). There's a tutorial for getting started with generating test suites. You'll also find tutorials for using Symflower with JUnit 4 and 5, finding reproducers, and using Symflower for TDD.

We have collected some of our favorite resources from the Symflower blog which provide plenty of valuable information on topics including LLMs, Spring Boot, Java, JUnit, TDD, and various programming principles and testing concepts.

Do I need a license to use Symflower?​

Yes, some of Symflower's features are only accessible with a license key. For pricing information and to obtain a license key, visit our Licensing page. See the Licensing section of this documentation to find out how to activate your license.

How long does it take for Symflower to generate tests?​

Smart test template generation usually only takes an instant, with test template creation for a whole file or repository generally not taking longer than a few seconds.

Since Generating test suites via symbolic execution requires thorough code analysis, it can take from a few seconds to minutes (or even more if you want to generate them for a whole repository) depending on the complexity of the analyzed code. Note that you can configure Symflower's settings to maximize the analysis time per method, ensuring that processes taking too long will be timed out.

The time it takes to generate test suites via LLMs depends on the configured LLM.

Does Symflower use LLMs to generate tests?​

Symflower offers two methods for generating test suites:

How accurate are the tests generated by LLMs?​

LLM-generated tests are dependent on the quality of the LLM that is configured. You should always validate the tests that are generated by LLMs.

How does Symflower's test generation use my data?​

Symflower's generate test via symbolic execution feature works 100% locally, meaning that it doesn't send home any of your code.

When generating tests via LLMs, Symflower does not use or store your data. Symflower's Test Generation via LLM feature forwards your code to the configured LLM's provider. The request is sent to the LLM and the response is written into a virtual file. Read the data policy of your selected LLM provider to understand how your data is used.

How do I manage generated tests?​

Symflower provides several features that make it easy to manage your tests. See the managing tests section of this documentation for details.

What are Symflower's limitations?​

Symflower's Generate test via symbolic execution feature currently supports Java, Spring, and Spring Boot applications (available for Java versions 1-17 and partly version 18-21). The tool doesn't currently support external dependencies or the full standard library, so it may not produce complete test suites for methods relying on either of these. We're constantly working to improve Symflower's test generation engine to make it useful in more and more scenarios. Have any suggestions or issues that you think should be fixed? Submit an issue or send us a message!

What build tools does Symflower support?​

Symflower currently supports Maven and Gradle.

What testing frameworks does Symflower support?​

Symflower works with both JUnit 4 and JUnit 5. Your project's configuration will be automatically detected, or you can just manually configure Symflower to use the testing framework you prefer. See the tutorial on using Symflower with JUnit 4 and JUnit 5 in this documentation for details.

What IDEs does Symflower support?​

Symflower is available as a plugin for VS Code, IntelliJ IDEs (IDEA, Android Studio, GoLand), and as a CLI tool.

I encountered a bug. What should I do now?​

Please submit a bug report via GitHub. See the Debugging section of this documentation for details on how to access Symflower's logs.

Note that you may experience bugs when using Symflower's test suite generation. We ask you to submit these issues to make sure we can fix them!

Can I use Symflower for Test-Driven Development?​

Yes. See the tutorial on using Symflower to support your TDD workflow in this documentation for details.

Can I use Symflower to find reproducers for debugging?​

Yes. See the tutorial for finding reproducers with Symflower in this documentation for details.

Does Symflower support mocking?​

Yes, Symflower uses Mockito to generate mocks automatically. Mocking is supported for both Java and Spring Boot applications.

How do I integrate Symflower into my workflow?​

To generate a smart test template or to generate a test suite, use the context menu, define your own hotkeys, or use Symflower via the command palette. For generating test templates, you can also use the code lens. For a seamless experience, configure Symflower to enable automated test generation upon saving the file.

Is there an online or cloud version of Symflower?​

Symflower is currently only available as a tool that runs locally on your device.

Are there any hardware and software (OS) requirements to use Symflower?​

Symflower currently supports the following architectures:

  • x86 with 64bit
  • ARM with 64bit

And operating systems:

  • Linux
  • MacOS
  • Windows