REVIEW 3 major objections 6 minor 36 references
ColorGo: Directed Concolic Execution
T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read ColorGo reaches target code sites and reproduces crashes up to 100x faster than directed greybox fuzzing by combining compilation-based concolic execution with incremental coloration.
desk verdict Promising directed concolic fuzzer with a real loop-handling gap that the evaluation doesn't cover; the headline speedups hold only on the selected targets. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is incremental coloration together with the Fast Depth First Search (FDFS) scheduling strategy. Static coloration, performed in an LLVM pass at compile time, marks the connected subgraph of the interprocedural control-flow graph that can reach the target; dynamic coloration removes infeasible subtrees at runtime when the path constraint cannot be satisfied. FDFS keeps newly generated inputs on a stack, terminates execution as soon as both successors of a branch are uncolored, and invokes the constraint solver only at deviation basic blocks, where one successor is colored and the other is not. Supporting machinery includes a partial function model that summarizes important C standard library functions as disjunctive return expressions, and reverse-edge stopping that prevents loop back edges from coloring the full loop body.
What would settle it
Build a small program whose only route to the target goes through an indirect call whose callee is chosen by input data, so the points-to analysis cannot resolve it; run ColorGo on that target. If the edge is not colored, every execution will terminate early at the uncolored successor and the tool will fail to reach the target even though a valid input exists.
Extended reading notes
Core claim
The paper's discovery, stated on its own terms, is that directed fuzzing can be cast as a one-source, multi-target graph search over the interprocedural control-flow graph, with the search space marked by coloration instead of heuristic distance. At compile time ColorGo colors every basic block that can reach the target, using backward propagation from target lines and target functions, points-to analysis for indirect calls, and a reverse-edge stop so loops do not color the whole loop body. At runtime it adds dynamic coloration by checking path-constraint feasibility and slicing away infeasible subtrees, then executes under a stack-based Fast Depth First Search that asks the constraint solver only at deviation basic blocks, where exactly one successor is colored. The reported results are time-to-reach speedups around 50x and time-to-expose speedups up to 100x over the baseline, with early termination ending every run outside the colored region on the evaluated vulnerability-reproduction tasks.
Load-bearing premise
ColorGo's speed depends on static coloration being complete: every basic block that can lead to the target must be marked as colored, because the runtime stops the whole search as soon as a branch has no colored successor, so one missed edge, such as through an unresolved indirect call, silently removes the only path to the target.
Editorial extensions
If this is right
- Directed fuzzing would no longer need to choose between random-mutation speed and symbolic precision; one native execution pass with targeted solver calls can serve both.
- Crash reproduction from a stack trace becomes practical for highly constrained checks such as input == 123456789, where random mutation would need enormous attempts.
- Because coloration is computed once at compile time, every subsequent execution and solver call is confined to the relevant region, shrinking time spent on irrelevant code.
- The approach requires recompilable source, but function models for common library calls extend it to programs that link uninstrumented libraries.
- The reported 100% early-termination rate on the vulnerability-reproduction targets indicates that search-scope restriction, not the solver alone, is what drives the speedup.
Reading between the lines
- A natural robustness extension would be a fallback that, when no colored path is found, resumes exploring uncolored edges so a single missed points-to edge cannot silently doom the search.
- FDFS's greedy stack discipline could be hybridized with distance-based seed prioritization from greybox fuzzing to handle large colored regions or multiple target sites.
- The evaluation runs each target once because the method is deterministic; varying the initial input queue and solver timeouts would reveal how stable the 50x to 100x margins are.
- The same coloration machinery could be reused for patch testing by feeding changed lines as target lines, since the framework already accepts arbitrary target lines and stack-trace-derived call chains.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes ColorGo, a directed whitebox fuzzer that combines compilation-based concolic execution (built on SymCC-style instrumentation) with an incremental coloration scheme. Static coloration marks, at compile time, basic blocks backward-reachable from a target site on the interprocedural control-flow graph (iCFG), stopping at reverse edges; dynamic coloration then prunes infeasible path constraints at runtime. The executor uses early termination on uncolored paths and a deviation-based search strategy (FDFS) that solves constraints only at branch points where the trace leaves the colored subgraph. The authors evaluate ColorGo on three real-world programs (jasper, lame, binutils cxxfilt) and report up to 50x faster time-to-reach and up to 100x faster time-to-expose compared to AFLGo, alongside an ablation study and an instrumentation-overhead measurement.
Significance. If the central claims hold, ColorGo would be a valuable step toward making directed fuzzing both precise and scalable: the compilation-based concolic approach promises near-native execution speed while retaining the input-generation precision of symbolic execution, and the coloration search-space restriction is a clean way to focus effort on relevant code. The system is implemented with modest code changes over SymCC, and the authors provide a publicly available artifact with reproduction guidelines, which strengthens verifiability. However, the significance is bounded by two issues: the static coloration is intentionally incomplete for loop-carried paths, and the empirical evaluation is thin (a single run of ColorGo, AFLGo results reported only as means, and how timeouts are treated is unclear). These issues do not negate the potential of the underlying idea, but they do limit the strength of the paper's headline claims until addressed.
major comments (3)
- [3.2 and 4.1] The static coloration stops backward propagation at reverse edges (Section 3.2: 'we stop the iterative coloration when detecting the reverse edge'; Section 4.1: 'we stop the propagation when detecting inverse edge'). This makes the colored iCFG incomplete for any target that is reachable only after loop-carried state updates. At a loop header, the back-edge successor is uncolored and the exit successor is colored, so the header is a deviation basic block; FDFS will solve for the exit condition and never explore the loop body. Thus a target such as a check after a loop on a counter or accumulated string can never be reached, because the runtime never attempts the uncolored body. The paper's loop example (Listing 2 / Figure 3) is reachable in a single iteration, so the evaluation does not exercise this case. Dynamic coloration only prunes infeasible constraints and never adds omitted edges, so the gap is not repaired. This is a load-bearing soundness limitation for the general claim of reaching target sites; the paper should either provide a mechanism to explore loop-carried paths (e.g., bounded re-coloring of back edges or a fallback to uncolored exploration) or explicitly restrict the claim to targets whose all feasible paths avoid cycle traversal.
- [5.2, Table 2] The headline '50x faster' time-to-reach result is not supported by the reported data. In Table 2, AFLGo times out (T.O.) on all three jasper targets, and the text says 'even discard data that AFLGo timeout (>24h)'. If timeouts are discarded, the mean TTR comparison is computed only over the three lame targets, so the 50x number is not a meaningful aggregate over the table. If timeouts are instead intended as censored observations, the summary statistic and its computation should be stated explicitly, and a survival-analysis or median-based comparison would be more appropriate. Moreover, ColorGo is executed once, while AFLGo is reported only as a mean over 10 runs without variance or confidence intervals. As stated, the data do not substantiate the claimed speedup.
- [5.3, Table 3] The '100x faster' time-to-expose claim is based on a small set of four CVEs, with ColorGo measured once and AFLGo reported only as means without distributions. For CVE-2016-4488, the comparison is not strongly in ColorGo's favor (927 ms vs. a reported 1 s mean), and the text dismisses this case rather than incorporating it into the analysis. A single deterministic run against a randomized baseline cannot establish a reliable performance ratio; the paper should report across multiple runs or justify why the deterministic execution is not subject to environmental noise. This is a central empirical claim and needs stronger statistical support.
minor comments (6)
- [3.4] The reference 'SMART [? ]' is an unresolved citation placeholder and must be completed or removed.
- [2.1] There are several typos: 'Backgroud' should be 'Background', 'venerability' should be 'vulnerability', and 'crush' should be 'crash' (Section 5.3).
- [5.5, Table 4] The stated overhead percentages (67%, 62%, 50%) do not obviously follow from Table 4; for example, the default (36 ms) vs. pure execution (16 ms) for CVE-2016-4487 implies about 125% overhead, not 67%. Please clarify the calculation and the quantities being compared.
- [5.2] The expression 'T.O' in Table 2 should be defined explicitly (timeout after 24 hours) and the handling of timeouts in the reported summary statistics should be described precisely, not only by a parenthetical remark.
- [5.1] The statement 'Our method does not contains randomness, and therefore statistical evaluation is unnecessary' is too strong; even deterministic tools can experience OS-level variability, and a single run is generally insufficient for a performance comparison. Please justify or soften this claim.
- [4.1] The phrase 'a feedback version by points-to-analysis [1]' is unclear; please clarify how the Anderson-style points-to analysis results are used to resolve indirect call targets in the coloration process.
Circularity Check
No significant circularity: target sites and stack traces are user inputs; measured speedups are comparisons against external baseline AFLGo; the self-cited artifact link is not load-bearing.
full rationale
ColorGo's derivation chain contains no circular step. Target lines and stack traces are user-supplied inputs, not outputs; Section 3.2 says static coloration takes as input target lines and the function call chain extracted from the stack trace, and the runtime then attempts to reach those exogenously defined sites. The claimed 50x/100x speedups are measured TTR/TTE comparisons against the external baseline AFLGo, and no parameter is fitted to evaluation outcomes: the paper states 'Our method does not contains randomness, and therefore statistical evaluation is unnecessary.' The sole self-citation ([7]) is the authors' artifact repository for data availability, not load-bearing evidence. Technical components are borrowed from external work (Windranger deviation basic blocks, SYMCC instrumentation, Andersen/SVF points-to analysis). No uniqueness theorem is imported and no known result is merely renamed. The reviewer concern that reverse-edge stopping (Sec. 3.2/4.1) omits loop-carried paths is a soundness limitation of static coloration, not circularity: the omission is a failure to reach the target, not a reduction of the method's output to its input. Therefore score 0.
Assumptions & free parameters
assumptions (5)
- domain assumption Compilation-based concolic execution runs close to native speed and scales to real programs.
- domain assumption Static coloration with debug info plus points-to analysis identifies all relevant code on the path to the target.
- ad hoc to paper Stopping backward coloration at reverse edges does not drop target-reachable paths that require extra loop iterations.
- domain assumption Modeling only selected C standard library functions preserves enough symbolic state for directed search.
- domain assumption AFLGo is a representative state-of-the-art baseline for the comparison.
Cite this review
Pith. "Pith review of ColorGo: Directed Concolic Execution." pith.science (2026). https://pith.science/paper/XLGJAWLL
@misc{pith2026250521130,
author = {Pith},
title = {Pith review of: ColorGo: Directed Concolic Execution},
year = {2026},
howpublished = {\url{https://pith.science/paper/XLGJAWLL}},
note = {Machine review of arXiv:2505.21130}
}
read the original abstract
Directed fuzzing is a critical technique in cybersecurity, targeting specific sections of a program. This approach is essential in various security-related domains such as crash reproduction, patch testing, and vulnerability detection. Despite its importance, current directed fuzzing methods exhibit a trade-off between efficiency and effectiveness. For instance, directed grey-box fuzzing, while efficient in generating fuzzing inputs, lacks sufficient precision. The low precision causes time wasted on executing code that cannot help reach the target site. Conversely, interpreter- or observer-based directed symbolic execution can produce high-quality inputs while incurring non-negligible runtime overhead. These limitations undermine the feasibility of directed fuzzers in real-world scenarios. To kill the birds of efficiency and effectiveness with one stone, in this paper, we involve compilation-based concolic execution into directed fuzzing and present ColorGo, achieving high scalability while preserving the high precision from symbolic execution. ColorGo is a new directed whitebox fuzzer that concretely executes the instrumented program with constraint-solving capability on generated input. It guides the exploration by \textit{incremental coloration}, including static reachability analysis and dynamic feasibility analysis. We evaluated ColorGo on diverse real-world programs and demonstrated that ColorGo outperforms AFLGo by up to \textbf{100x} in reaching target sites and reproducing target crashes.
Figures
Figures from the paper (1 more)
Reference graph
Works this paper leans on
-
[1]
Program analysis and specialization for the c programming language
Lars Ole Andersen. Program analysis and specialization for the c programming language. 1994
work page 1994
-
[2]
Directed greybox fuzzing with afl
Marcel Böhme. Directed greybox fuzzing with afl. https://github.com/aflgo/aflgo, 2021
work page 2021
-
[3]
Marcel Böhme, Van-Thuan Pham, Manh-Dung Nguyen, and Abhik Roychoudhury. Directed greybox fuzzing. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security , pages 2329–2344, 2017
work page 2017
-
[4]
Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs
Cristian Cadar, Daniel Dunbar, Dawson R Engler, et al. Klee: Unassisted and automatic generation of high-coverage tests for complex systems programs. In OSDI, volume 8, pages 209–224, 2008
2008
-
[5]
Hawkeye: Towards a desired directed grey-box fuzzer
Hongxu Chen, Yinxing Xue, Yuekang Li, Bihuan Chen, Xiaofei Xie, Xiuheng Wu, and Yang Liu. Hawkeye: Towards a desired directed grey-box fuzzer. In Proceedings of the 2018 ACM SIGSAC conference on computer and communications security, pages 2095–2108, 2018
work page 2018
-
[6]
Savior: Towards bug-driven hybrid testing
Yaohui Chen, Peng Li, Jun Xu, Shengjian Guo, Rundong Zhou, Yulong Zhang, Tao Wei, and Long Lu. Savior: Towards bug-driven hybrid testing. In 2020 IEEE Symposium on Security and Privacy (SP) , pages 1580–1596. IEEE, 2020
work page 2020
-
[7]
The repository of our evaluation artifacts
ColorGo. The repository of our evaluation artifacts. https://github.com/lijia73/ ColorGo, 2023
work page 2023
-
[8]
Symfusion: hybrid instrumen- tation for concolic execution
Emilio Coppa, Heng Yin, and Camil Demetrescu. Symfusion: hybrid instrumen- tation for concolic execution. In Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering , pages 1–12, 2022
work page 2022
Show all 36 references
-
[9]
Use containers to build, share and run your applications
Docker. Use containers to build, share and run your applications. https://www. docker.com/resources/what-container, 2021
2021
-
[10]
Windranger: a directed greybox fuzzer driven by deviation basic blocks
Zhengjie Du, Yuekang Li, Yang Liu, and Bing Mao. Windranger: a directed greybox fuzzer driven by deviation basic blocks. In Proceedings of the 44th International Conference on Software Engineering , pages 2440–2451, 2022
2022
-
[11]
Dart: Directed automated ran- dom testing
Patrice Godefroid, Nils Klarlund, and Koushik Sen. Dart: Directed automated ran- dom testing. In Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation , pages 213–223, 2005
2005
-
[12]
Beacon: Directed grey-box fuzzing with provable path pruning
Heqing Huang, Yiyuan Guo, Qingkai Shi, Peisen Yao, Rongxin Wu, and Charles Zhang. Beacon: Directed grey-box fuzzing with provable path pruning. In 2022 IEEE Symposium on Security and Privacy (SP) , pages 36–50. IEEE, 2022
2022
-
[13]
Tiff: Using input type inference to improve fuzzing
Vivek Jain, Sanjay Rawat, Cristiano Giuffrida, and Herbert Bos. Tiff: Using input type inference to improve fuzzing. In Proceedings of the 34th Annual Computer Security Applications Conference, pages 505–517, 2018
2018
-
[14]
Bugredux: Reproducing field failures for in-house debugging
Wei Jin and Alessandro Orso. Bugredux: Reproducing field failures for in-house debugging. In 2012 34th international conference on software engineering (ICSE) , pages 474–484. IEEE, 2012
2012
-
[15]
Directing a search towards execution properties with a learned fitness function
Leonid Joffe and David Clark. Directing a search towards execution properties with a learned fitness function. In 2019 12th IEEE Conference on Software Testing, Validation and Verification (ICST), pages 206–216. IEEE, 2019
2019
-
[16]
Poster: Directed hybrid fuzzing on binary code
Juhwan Kim and Joobeom Yun. Poster: Directed hybrid fuzzing on binary code. In Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pages 2637–2639, 2019
2019
-
[17]
In 30th USENIX Security Symposium (USENIX Security 21) , pages 2777–2794, 2021
Yuwei Li, Shouling Ji, Yuan Chen, Sizhuang Liang, Wei-Han Lee, Yueyao Chen, Chenyang Lyu, Chunming Wu, Raheem Beyah, Peng Cheng, et al.{UNIFUZZ}: A holistic and pragmatic{Metrics-Driven} platform for evaluating fuzzers. In 30th USENIX Security Symposium (USENIX Security 21) , ...
2021
-
[18]
V-fuzz: Vulnerability prediction-assisted evolutionary fuzzing for binary programs
Yuwei Li, Shouling Ji, Chenyang Lyu, Yuan Chen, Jianhai Chen, Qinchen Gu, Chunming Wu, and Raheem Beyah. V-fuzz: Vulnerability prediction-assisted evolutionary fuzzing for binary programs. IEEE Transactions on Cybernetics , 52(5):3745–3756, 2020
2020
-
[19]
Sequence directed hybrid fuzzing
Hongliang Liang, Lin Jiang, Lu Ai, and Jinyi Wei. Sequence directed hybrid fuzzing. In 2020 IEEE 27th International Conference on Software Analysis, Evolution and Reengineering (SANER), pages 127–137. IEEE, 2020
2020
-
[20]
Sequence cov- erage directed greybox fuzzing
Hongliang Liang, Yini Zhang, Yue Yu, Zhuosi Xie, and Lin Jiang. Sequence cov- erage directed greybox fuzzing. In 2019 IEEE/ACM 27th International Conference on Program Comprehension (ICPC) , pages 249–259. IEEE Computer Society, 2019
2019
-
[21]
Selectfuzz: Efficient directed fuzzing with selective path exploration
Changhua Luo, Wei Meng, and Penghui Li. Selectfuzz: Efficient directed fuzzing with selective path exploration. In 2023 IEEE Symposium on Security and Privacy 10 ColorGo: Directed Concolic Execution Conference’17, July 2017, Washington, DC, USA (SP), pages 2693–2707. IEEE, 2023
2023
-
[22]
Katch: High-coverage testing of software patches
Paul Dan Marinescu and Cristian Cadar. Katch: High-coverage testing of software patches. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, pages 235–245, 2013
2013
-
[23]
Binary-level directed fuzzing for{Use-After-Free} vulnera- bilities
Manh-Dung Nguyen, Sébastien Bardin, Richard Bonichon, Roland Groz, and Matthieu Lemerre. Binary-level directed fuzzing for{Use-After-Free} vulnera- bilities. In 23rd International Symposium on Research in Attacks, Intrusions and Defenses (RAID 2020), pages 47–62, 2020
2020
-
[24]
Badger: complexity analysis with fuzzing and symbolic execution
Yannic Noller, Rody Kersten, and Corina S Păsăreanu. Badger: complexity analysis with fuzzing and symbolic execution. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis , pages 322–332, 2018
2018
-
[25]
Hydiff: Hybrid differential software analysis
Yannic Noller, Corina S Păsăreanu, Marcel Böhme, Youcheng Sun, Hoang Lam Nguyen, and Lars Grunske. Hydiff: Hybrid differential software analysis. In Pro- ceedings of the ACM/IEEE 42nd International Conference on Software Engineering , pages 1273–1285, 2020
2020
-
[26]
Slowfuzz: Automated domain-independent detection of algorithmic complexity vulner- abilities
Theofilos Petsios, Jason Zhao, Angelos D Keromytis, and Suman Jana. Slowfuzz: Automated domain-independent detection of algorithmic complexity vulner- abilities. In Proceedings of the 2017 ACM SIGSAC conference on computer and communications security, pages 2155–2168, 2017
2017
-
[27]
Energy distribution matters in greybox fuzzing
Lingyun Situ, Linzhang Wang, Xuandong Li, Le Guan, Wenhui Zhang, and Peng Liu. Energy distribution matters in greybox fuzzing. In 2019 IEEE/ACM 41st International Conference on Software Engineering: Companion Proceedings (ICSE- Companion), pages 270–271. IEEE, 2019
2019
-
[28]
Svf: interprocedural static value-flow analysis in llvm
Yulei Sui and Jingling Xue. Svf: interprocedural static value-flow analysis in llvm. In Proceedings of the 25th international conference on compiler construction , pages 265–266, 2016
2016
-
[29]
Typestate-guided fuzzer for discovering use- after-free vulnerabilities
Haijun Wang, Xiaofei Xie, Yi Li, Cheng Wen, Yuekang Li, Yang Liu, Shengchao Qin, Hongxu Chen, and Yulei Sui. Typestate-guided fuzzer for discovering use- after-free vulnerabilities. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering , pages 99...
2020
-
[30]
Tofu: Target-oriented fuzzer
Zi Wang, Ben Liblit, and Thomas Reps. Tofu: Target-oriented fuzzer. arXiv preprint arXiv:2004.14375, 2020
2004 arXiv
-
[31]
Memlock: Memory usage guided fuzzing
Cheng Wen, Haijun Wang, Yuekang Li, Shengchao Qin, Yang Liu, Zhiwu Xu, Hongxu Chen, Xiaofei Xie, Geguang Pu, and Ting Liu. Memlock: Memory usage guided fuzzing. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, pages 765–777, 2020
2020
-
[32]
Rdfuzz: Accelerating directed fuzzing with intertwined schedule and optimized mutation
Jiaxi Ye, Ruilin Li, and Bin Zhang. Rdfuzz: Accelerating directed fuzzing with intertwined schedule and optimized mutation. Mathematical Problems in Engi- neering, 2020:1–12, 2020
2020
-
[33]
Semfuzz: Semantics-based automatic generation of proof-of-concept exploits
Wei You, Peiyuan Zong, Kai Chen, XiaoFeng Wang, Xiaojing Liao, Pan Bian, and Bin Liang. Semfuzz: Semantics-based automatic generation of proof-of-concept exploits. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pages 2139–2154, 2017
2017
-
[34]
Suzzer: A vulnerability- guided fuzzer based on deep learning
Yuyue Zhao, Yangyang Li, Tengfei Yang, and Haiyong Xie. Suzzer: A vulnerability- guided fuzzer based on deep learning. In International Conference on Information Security and Cryptology, pages 134–153. Springer, 2020
2020
-
[35]
Regression greybox fuzzing
Xiaogang Zhu and Marcel Böhme. Regression greybox fuzzing. In Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security , pages 2169–2182, 2021
2021
-
[36]
In 29th USENIX security symposium (USENIX security 20) , pages 2255–2269, 2020
Peiyuan Zong, Tao Lv, Dawei Wang, Zizhuang Deng, Ruigang Liang, and Kai Chen.{FuzzGuard}: Filtering out unreachable inputs in directed grey-box fuzzing through deep learning. In 29th USENIX security symposium (USENIX security 20) , pages 2255–2269, 2020. 11
2020
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.