Report Number: CSM-262 Report Title: Comparing CSP algorithms without considering variable ordering heuristics can be misleading Authors: Alvin C. M. Kwan & Edward P. K. Tsang Abstract Many new algorithms for solving constraint satisfaction problems have been proposed in recent years. Their performance is usually compared with some other algorithms by measuring the number of compatibility checks or the cpu time that the algorithms need for solving a predetermined set of problems. There are some studies which compare algorithms without using any variable ordering heuristics. We argue that evaluating algorithms without considering their combinations with variable orering heuristic could produce misleading results. We advocate that algorithm-heuristic combinations should be compared instead of algorithms alone.