Your work is excellent and thanks for releasing the code. However when excuting your code I face with a problem. I trained 1 mode 5w1s MAML on cifar by:
python main.py --dataset cifar --maml-model True --num-batches 600000 --output-folder maml_5w1s_cifar
The training performance is fine with "accuracy meta_val after" > 80%, but I cannot obtain similar result in --eval mode:
python main.py --dataset cifar --maml-model True --num-batches 60000 --checkpoint ./train_dir/maml_5w1s_cifar/maml_conv_60000.pt --eval
The "accuracy meta_val after" is about just 30%, and confidence interval gives similar result on "post_val_accuracy":
Evaluation results:
num_batches: 6000
post_train_accuracy: 0.9916700004965068 +- 0.0003492838883183404
post_train_loss: 0.4392507508163651 +- 0.0006438929325898017
post_val_accuracy: 0.28511111702825875 +- 0.0006488388304810349
post_val_loss: 1.6567623161733154 +- 0.0012289125473787653
pre_train_accuracy: 0.20061333698555828 +- 0.0013524214528860354
pre_train_loss: 1.6136694465855759 +- 0.0002620773080873398
pre_val_accuracy: 0.20086889288052914 +- 0.0005964522753135613
pre_val_loss: 1.6139000959157945 +- 0.00012502413922341256
Is 28.51%+- 0.06% the accuracy of MAML on cifar? I am new to meta learning and I doubt whether I use your code properly. I would really appreciate it if you could help me evaluate the model in correct way. Thanks!
In addition, problem also occurs in 2w1s training on cifar, but does not occur on omniglot dataset.
Your work is excellent and thanks for releasing the code. However when excuting your code I face with a problem. I trained 1 mode 5w1s MAML on cifar by:
python main.py --dataset cifar --maml-model True --num-batches 600000 --output-folder maml_5w1s_cifarThe training performance is fine with "accuracy meta_val after" > 80%, but I cannot obtain similar result in --eval mode:
python main.py --dataset cifar --maml-model True --num-batches 60000 --checkpoint ./train_dir/maml_5w1s_cifar/maml_conv_60000.pt --evalThe "accuracy meta_val after" is about just 30%, and confidence interval gives similar result on "post_val_accuracy":
Is 28.51%+- 0.06% the accuracy of MAML on cifar? I am new to meta learning and I doubt whether I use your code properly. I would really appreciate it if you could help me evaluate the model in correct way. Thanks!
In addition, problem also occurs in 2w1s training on cifar, but does not occur on omniglot dataset.