@@ -11,19 +11,34 @@ Framework for evaluating causal inference methods.
1111 - [ Authors] ( #authors )
1212
1313## General
14- Causality-Benchmark is a library developed by IBM Research for benchmarking algorithms that
15- estimate causal effect.
16- The framework includes unlabeled data, labeled data, and code for scoring algorithm predictions.
17- It can benchmark predictions of both population effect size and individual effect size.
18-
19- The evaluation script is not bounded to the provided data, and can be used on other data as
20- long as some basic requirements are kept regarding the formats.
21- For more technical details about the evaluation metrics and the data, please refer to the
22- framework menuscript ** TODO: Link to the menuscript/technical report**
23-
24- Please note that due to GitHub limitation, only a sample of the data is available in this
25- repository. However, you can manually access and download the entire dataset from the
26- [ Synapse sharing platform] ( https://www.synapse.org/#!Synapse:syn11294478/files/ )
14+ Causality-Benchmark is a library developed by IBM Research Haifa for
15+ benchmarking algorithms that estimate the causal effect of a treatment on
16+ some outcome. The framework includes unlabeled data, labeled data, and code
17+ for scoring algorithm predictions. It can benchmark predictions of both
18+ population effect size and individual effect size.
19+
20+ The feature matrix is derived from the
21+ [ linked birth and infant death data] ( https://www.cdc.gov/nchs/nvss/linked-birth.htm ) ,
22+ and the labeled and unlabeled data are based on simulated models of the
23+ treatment assignment, treatment effect, and censoring.
24+
25+ The evaluation script is not bounded to the provided data,
26+ and can be used on other data as
27+ long as some basic requirements are kept regarding the formats.
28+ Full technical details regarding the calculated metrics and the formats of the
29+ labeled and unlabeled data will be published soon through a related manuscript.
30+ Meanwhile, most of the details can be found in the
31+ [ 2018 Casual Inference Challenge] ( https://www.synapse.org/ACIC2018Challenge )
32+ website.
33+
34+ Please note that due to GitHub limitation, only a sample of the data is
35+ available in this repository. However, you can manually access and download
36+ the entire dataset from the
37+ [ Synapse sharing platform] ( https://www.synapse.org/#!Synapse:syn11294478/files/ ) .
38+ Furthermore, since the benchmarking tool is used in the
39+ [ Casual Inference Challenge 2018] ( https://www.synapse.org/#!Synapse:syn11294478 ) ,
40+ the dataset currently includes a handful of example data with labels.
41+ The full set of labeled data will be available when the challenge ends.
2742
2843## Getting Started
2944### Prerequisites
@@ -82,7 +97,7 @@ scores = evaluate(PATH_TO_PREDICTION_OUTPUT, PATH_TO_COUNTERFACTUAL_FILES_DIRECT
8297 individual_prediction = True )
8398```
8499##### Expected Files
85- * The counterfactual files (holding $y^1$, $y^0$ for each individual), are expected to be a
100+ * The counterfactual outcomes files (holding $y^1$, $y^0$ for each individual), are expected to be a
86101 directory with different comma-separated-files and their file names corresponding to the
87102 data-instance but having some suffix (e.g. ` "_cf.csv" ` ).
88103* The predictions for population effect size are expected to be one comma-delimited-file with
@@ -91,7 +106,7 @@ scores = evaluate(PATH_TO_PREDICTION_OUTPUT, PATH_TO_COUNTERFACTUAL_FILES_DIRECT
91106 comma-delimited-files, each corresponding to a data-instance and each containing the
92107 estimated outcome under no-treatment and under positive treatment.
93108
94- For full explanation, please refer to the menuscript ** TODO: link to menuscript **
109+ For full explanation, please refer to the menuscript.
95110
96111#### Estimation
97112To avoid inflating file sizes for nothing,
@@ -128,10 +143,7 @@ The current content is open source under Apache License 2.0. For full specificat
128143[ License.txt] ( License.txt )
129144
130145## Authors
131- * bullets (link to personal github profile)
132- * of
133- * authors' (link to personal site)
134- * names
135-
136-
146+ * Yishai Shimoni ([ Homepage] ( http://researcher.watson.ibm.com/researcher/view.php?person=il-YISHAIS ) )
147+ * Chen Yanover ([ Homepage] ( http://researcher.watson.ibm.com/researcher/view.php?person=il-CHENY ) )
148+ * Ehud Karavani ([ Github] ( https://github.com/ehudkr ) )
137149
0 commit comments