AE++
This commit is contained in:
parent
0ee6f02a89
commit
045709877e
21
AE.md
21
AE.md
@ -10,6 +10,7 @@ This document provides instructions for reproducing the experimental results fro
|
||||
|
||||
### Claims Supported by Artifact
|
||||
If you run the benchmarks as described below, it will produce the following files, which correspond to figures in the paper.
|
||||
|
||||
1. **Figure 3**: (Files: `sql_waters_seq_bytes`, `sql_polycopter_seq_dataflow_full`) This scenario is about mutating just input values. While multiple techniques find the worst case for both scenarios, FRET is the fastest to reach the maximum, particular in the second case. FRET also achieves the highest median result.
|
||||
2. **Figure 4**: (Files: `sql_waters_seq_int`,`sql_release_seq_int`) This scenario is about mutating just interrupt times. While multiple techniques find the worst case for the second scenario, FRET achieves the highest response time on the first one.
|
||||
3. **Figure 5**: (Files: `sql_release_seq_full`, `sql_waters_seq_full`) This scenario is about mutating both kinds of inputs simultaniously. Only FRET achieves the worst possible time when looking at the median results of the first scenario. For the second one, FRET alone reaches the highest response times. Thid demonstrates, that FRET's advantage over other techniques is particularly pronounced when both kinds of inputs are compared.
|
||||
@ -19,6 +20,7 @@ If you run the benchmarks as described below, it will produce the following file
|
||||
|
||||
### Option 1: VirtualBox Images (Recommended)
|
||||
Download our ready-made VM image from [https://sys-sideshow.cs.tu-dortmund.de/downloads/rtss25/fret.ova](https://sys-sideshow.cs.tu-dortmund.de/downloads/rtss25/fret.ova)
|
||||
|
||||
- **VM Configuration**: Allocate as much RAM as possible (minimum 32GB, recommended 512GB)
|
||||
- **CPU Allocation**: One core per 4-8GB of RAM (recommended 64 cores, 256-512GB RAM)
|
||||
- **Disk Space**: At least 100GB free space for results
|
||||
@ -128,6 +130,7 @@ open $DUMP/show_job.html
|
||||
```
|
||||
This script will reproduce all figures 3-6 in the eval section of the paper.
|
||||
You can edit the environemnt variables at the top to change the following parameters:
|
||||
|
||||
- `CORES`: The number of (physical) cores of the VM / fuzzers running in parallel. You will need about 8GB of RAM per fuzzer.
|
||||
- `RUNTIME`: Time spent on each fuzzing run in seconds (the default is 24h). 8h should be sufficient to see results similar to the paper.
|
||||
- `TARGET_REPLICA_NUMBER`: The number of replicas for each regular configuration.
|
||||
@ -147,6 +150,7 @@ export MULTIJOB_REPLICA_NUMBER=3
|
||||
## Results
|
||||
All results can be found in `LibAFL/fuzzers/FRET/benchmark` inside a directory `eval_xx-xx-xx`.
|
||||
The content should be the following:
|
||||
|
||||
- Plots inside the top-level of the directory. You can compare them to the paper according to the hints under "Claims Supported by Artifact" above.
|
||||
- A directory `timedump`, which contains subdirectories for each fuzzer.
|
||||
- Insude you find files for configuration with different seeds.
|
||||
@ -163,7 +167,7 @@ An archive of our results is also provided under [https://sys-sideshow.cs.tu-dor
|
||||
|
||||
#### Out of Memory Errors
|
||||
- Reduce `CORES` parameter to match available RAM (up to 8GB per core)
|
||||
- Consider using the quick evaluation configuration
|
||||
- Consider using fewer relicas per configuration
|
||||
|
||||
#### Build Failures
|
||||
```bash
|
||||
@ -192,14 +196,13 @@ nix-shell
|
||||
### Directory Layout
|
||||
```
|
||||
FRET/
|
||||
├── LibAFL/fuzzers/FRET/ # Main FRET fuzzer implementation
|
||||
│ ├── src/ # Source code
|
||||
│ ├── benchmark/ # Evaluation framework
|
||||
│ └── target/ # Compiled binaries
|
||||
├── FreeRTOS/FreeRTOS/Demo/CORTEX_M3_MPS2_QEMU_GCC # Our FreeRTOS Demos
|
||||
├── one_time_setup.sh # Initial setup script
|
||||
├── run_eval.sh # Main evaluation script
|
||||
└── AE.md # This document
|
||||
+-- LibAFL/fuzzers/FRET/ # Main FRET fuzzer implementation
|
||||
| +-- src/ # Source code
|
||||
| +-- benchmark/ # Evaluation framework
|
||||
+-- FreeRTOS/FreeRTOS/Demo/CORTEX_M3_MPS2_QEMU_GCC # Our FreeRTOS Demos
|
||||
+-- one_time_setup.sh # Initial setup script
|
||||
+-- run_eval.sh # Main evaluation script
|
||||
+-- AE.md # This document
|
||||
```
|
||||
|
||||
### Key Components
|
||||
|
Loading…
x
Reference in New Issue
Block a user