Question about the results of the attack

[milumilum]

hi，
when I experimented with the 1000 samples you provided from the AG dataset and the bert model, the results are:
For dataset data/ag: For target model bert: original accuracy: 94.200%, adv accuracy: 29.500%, avg changed rate: 25.076%, num of queries: 481.7
But the adv accuracy in your paper is 11.5%.
I'm not quite sure what led to this result. So did you use the same 1000 examples in your paper?

Thanks.

[jind11]

I am also not sure why there is such a huge change. Did yo use the same BERT model parameters that I provided?

[milumilum]

Yes, the model parameters and data are all provided by you.All the other attacks are close to the results given in your paper, only this one deviates a lot, so I wonder whether it is due to the data.

[Youoo1]

Where is the emdding.npz file, please? Or how is it generated?