commit 9b50450c4c65b16813fb8187174b72b7992d5f58 · dekker.one/on-restart-benchmarks

Pipfile

···

       7
       7
        
       numpy = "*"

     

       8
       8
        
       seaborn = "*"

     

       9
       9
        
       matplotlib = "*"

     

       10
       10
       +
       cffi = "*"

     

       10
       11
        
       

     

       11
       12
        
       [dev-packages]

     

       12
       13

+28 -19

README.md

···

       1
       1
       -
       # Restart Based Meta-Search Experiments

     

       1
       1
       +
       # Incremental Meta-Search Experiments

     

       2
       2
        
       

     

       3
       3
       -
       This repository contains benchmarks of restart based meta-search presented in

     

       3
       3
       +
       This repository contains benchmarks of meta-search through incremental usage of MiniZinc, included in Jip J. Dekker's PhD Thesis and partially presented in the following paper.

     

       4
       4
        
       

     

       5
       5
        
       > Dekker J.J., de la Banda M.G., Schutt A., Stuckey P.J., Tack G. (2018) Solver-Independent Large Neighbourhood Search. In: Hooker J. (eds) Principles and Practice of Constraint Programming. CP 2018. Lecture Notes in Computer Science, vol 11008. Springer, Cham. https://doi.org/10.1007/978-3-319-98334-9_6

     

       6
       6
        
       

     

       7
       7
       -
       and included in Jip J. Dekker's PhD Thesis.

     

       8
       8
       -
       

     

       9
       9
       -
       These experiments concern the usage of propagation engine adjustments that capture some of the context of the current search space.

     

       10
       10
       -
       This context can later be accessed by special propagators to enforce properties that are interesting for meta-search.

     

       11
       11
       -
       Such as:

     

       7
       7
       +
       These experiments concern two model for the incremental solving of MiniZinc models:

     

       8
       8
       +
       - The usage of propagation engine adjustments that capture some of the context of the current search space. This context can later be accessed by special propagators to enforce properties that are interesting for meta-search.  Such as:

     

       9
       9
       +
       	- The last value a variable has taken

     

       10
       10
       +
       	- The last solution a variable has taken

     

       11
       11
       +
       	- A random value, to be assigned by a variable

     

       12
       12
       +
       - An incremental rewriting engine for MiniZinc instances that minimizes overhead of incremental changes.

     

       12
       13
        
       

     

       13
       13
       -
       - The last value a variable has taken

     

       14
       14
       -
       - The last solution a variable has taken

     

       15
       15
       -
       - A random value, to be assigned by a variable

     

       16
       16
       -
       

     

       17
       17
       -
       These benchmarks tests the effectiveness of this approach.

     

       18
       18
       -
       The experiments are conducted on the `gbac`, `rcpsp-wet` and `steelmillslab` problems from the MiniZinc challenge.

     

       19
       19
       -
       Each model is given two neighbourhoods for an round-robin LNS approach.

     

       14
       14
       +
       These benchmarks tests the effectiveness of these approaches.

     

       15
       15
       +
       The experiments are conducted on the `gbac`, `radiation`, `rcpsp-wet` and `steelmillslab` problems from the MiniZinc challenge.

     

       16
       16
       +
       The `radiation` model uses a lexicographic objective.

     

       17
       17
       +
       The other models are solved using two neighbourhoods for an round robin LNS approach.

     

       20
       18
        
       

     

       21
       19
        
       ## Setup

     

       22
       20
        
       

     
···

       24
       22
        
       

     

       25
       23
        
       - The amount of time given for the test (120 sec)

     

       26
       24
        
       - The amount of repeats for randomised search (10)

     

       27
       27
       -
       - It adds the correect version of MiniZinc to the `PATH`

     

       25
       25
       +
       - It adds the correct version of MiniZinc to the `PATH`

     

       28
       26
        
       

     

       29
       27
        
       The solver versions used are stored in the `software/` folder as git sub-modules.

     

       30
       28
        
       The `install.sh` script can be used to compile the solvers and put them in the correct place.

     

       31
       29
        
       

     

       32
       30
        
       ## Run

     

       31
       31
       +
       

     

       32
       32
       +
       ### Large Neighbourhood Search

     

       33
       33
        
       

     

       34
       34
        
       The experiments can be run as different parts:

     

       35
       35
        
       

     

       36
       36
       -
       - Regular Gecode (base-line)

     

       37
       37
       -
       - Regular Chuffed (base-line)

     

       36
       36
       +
       - Regular Gecode (baseline)

     

       37
       37
       +
       - Regular Chuffed (baseline)

     

       38
       38
        
       - Restart Based Gecode (proposal)

     

       39
       39
        
       - Restart Based Chuffed (proposal)

     

       40
       40
        
       - Recording Gecode (data for replay)

     

       41
       41
       -
       - Replaying Gecode (ideal-case, no-computation)

     

       41
       41
       +
       - Replaying Gecode (ideal case, no computation)

     

       42
       42
        
       

     

       43
       43
        
       For each part there is a `run_*.sh` script that will run the experiments and put its results into the `output` folder.

     

       44
       44
        
       The `run_all.sh` script can be used to run them all in sequences.

     

       45
       45
        
       

     

       46
       46
       +
       ### Effort Comparison

     

       47
       47
       +
       

     

       46
       48
        
       ## Analyse

     

       47
       49
        
       

     

       50
       50
       +
       ### Large Neighbourhood Search

     

       51
       51
       +
       

     

       48
       52
        
       The scripts `analyse_{gecode,chuffed}.py` can be used to process the information gathered during the experiments.

     

       49
       53
        
       Each script is specific to that solver.

     

       50
       50
       -
       It takes the output folder of one of the runs as an argument (e.g., `output/gecode/gbac/original`).

     

       54
       54
       +
       It takes the output folder of one of the models as an argument (e.g., `output/gecode/gbac`).

     

       51
       55
        
       The script produces a CSV format that can then be processed using software like Excel or Numbers.

     

       56
       56
       +
       

     

       57
       57
       +
       The script `cumulative_analysis.py` can generate a PDF plot `output.pdf` for of the cumulative objective value over time.

     

       58
       58
       +
       It takes the output folder of one of the models and the name of the solver as an argument.

     

       59
       59
       +
       

     

       60
       60
       +
       ### Effort Comparison