​
Key Features

The system currently supports four main functionalities:

  1. Comprehensive Repository Understanding
    Automatically analyzes a GitHub repository and provides high-level summaries and usage instructions.
  2. Unit Test Execution for New PRs
    Automatically runs unit tests related to code changes in new pull requests.
  3. Unit Test Coverage Evaluation for New PRs
    Reviews code changes in a pull request, evaluates whether all necessary cases are covered by existing unit tests, and suggests improvements if needed.
  4. GitHub Documentation Consistency Checking
    Checks if documentation files (e.g., README, API docs) related to a PR have become out-of-date due to recent changes and suggests necessary updates.

​
Getting Started

​
1. Clone Repositories

First, clone both required repositories:

git clone https://github.com/Coral-Protocol/coral-server.git
git clone https://github.com/Coral-Protocol/software-testing-agents.git

​
2. Set Up the Python Environment

It is highly recommended to use a virtual environment (venv or conda):

With venv:

python3.10 -m venv venv
source venv/bin/activate

Or with conda:

conda create -n coral-agents python=3.10
conda activate coral-agents

​
3. Install Dependencies

Install all required packages:

pip install PyGithub
pip install langchain-mcp-adapters langchain-openai langchain langchain-core
pip install crewai
pip install 'camel-ai[all]'

​
4. Configure Environment Variables

Export your OPENAI_API_KEY and GITHUB_ACCESS_TOKEN:

export OPENAI_API_KEY=sk-xxx
export GITHUB_ACCESS_TOKEN=ghp_xxx

​
5. Start the MCP Server

Navigate to the coral-server directory and run:

cd coral-server
./gradlew run

Note: Gradle may appear to stall at 83%, but the server is running. Check the terminal logs to confirm.

​
6. Launch Agents (in seven separate terminals)

Navigate to the software-testing-agents directory and open seven terminals and run the following scripts in the software-testing-agents directory:

cd ..
cd software-testing-agents
python 0-langchain-interface.py
python 1-crewai-GitCloneAgent.py
python 2-camel-CodeDiffReviewAgent.py
python 3-langchain-UnitTestRunnerAgent.py
python 4-langchain-RepoUnderstandingAgent.py
python 5-langchain-RepoUnitTestAdvisorAgent.py
python 6-langchain-RepoDocConsistencyCheckerAgent.py

​
Usage Examples

​
1. Repository Understanding

Ask the Interface Agent for a comprehensive summary of a repository:

Please give me a comprehensive instruction of Coral-Protocol/coraliser.

Or specify a branch:

Please give me a comprehensive instruction of the master branch of Coral-Protocol/coral-server.

Or for other public repositories:

Please give me a comprehensive instruction of the master branch of camel-ai/camel.

🎬 Watch Video Demo

​
2. Unit Test Execution for New PRs

Ask the system to execute all relevant unit tests for a PR:

Please execute the unit test for the '6' PR in repo 'renxinxing123/software-testing-code'.
Please execute the unit test for the '2' PR in repo 'renxinxing123/camel-software-testing'.

🎬 Watch Video Demo

​
3. Unit Test Coverage Evaluation

Ask the system to evaluate whether a PR’s changes are fully covered by tests:

I created a new branch, `new-semantic-scholar-toolkit`, in the repository 'renxinxing123/camel-software-testing' and opened a new pull request (#3). For the changed files, could you please help me check whether the corresponding unit tests fully cover all necessary cases? Are there any additional tests that should be added?

🎬 Watch Video Demo

​
4. Documentation Consistency Checking

Ask the system to automatically check whether relevant documentation files are up-to-date with PR changes:

I created a new branch 'repo-understanding+unit-test-advice' in the repo 'renxinxing123/software-testing-agents-test' and opened a new PR (#2), could you please help me check if the relevant doc covered all the changes from the PR?

🎬 Watch Video Demo

​
Notes

  • When running tests, the system identifies the relevant unit test files and executes all test cases within them for reliable coverage.
  • Documentation consistency checking will analyze README, API docs, and other related files that may be affected by a PR and flag any outdated or missing documentation.
  • The project is designed for easy extension—feel free to add new agent scripts or tools!
  • Switching CodeDiffReviewAgent to Groq Llama 3 70B may help reduce cost and increase speed, but the quality of system coverage might be slightly affected compared to GPT-4.1.

👉 Check out the full implementation on GitHub: Coral-Protocol/software-testing-agents