public class BanditTrainer extends Object
The output indicates the quantiles of the distribution for regret relative to the optimal pick. The regret distribution is estimated by picking two random conversion probabilities and then running the betaBayesian model for a number of steps. The regret is computed by taking the expectation for the optimal choice and subtracting from the actual percentage of conversion achieved. On average, this should be somewhat negative since the model has to spend some effort examining the suboptimal choice. The median, 25 and 75%ile marks all scale downward fairly precisely with the square root of the number of trials which is to be expected from theoretical considerations.
The betaBayesian model works by keeping an estimate of the posterior distribution for the conversion probability for each of the bandits. We take a uniform distribution as the prior so the posterior is a beta distribution. The model samples probabilities from the two posterior distributions and chooses the model whose sample is larger. As data is collected for the two bandits, the better of the bandits will quickly have a pretty narrow posterior distribution and the lesser bandit will rarely have a sampled probability higher than the better bandit. This means that we will stop getting data from the less bandit, but only when there is essentially no chance that it is better.
Constructor and Description 

BanditTrainer() 
Modifier and Type  Method and Description 

static double 
averageRegret(String outputFile,
int[] sizes,
int replications,
int bandits)
Computes average regret relative to perfect knowledge given uniform random probabilities.

static double 
commitTime(String outputFile,
int n,
double p1,
double p2,
int cutoff)
Records which bandit was chosen for many runs of the same scenario.

static void 
main(String[] args) 
static double 
totalRegret(String cumulativeOutput,
String perTurnOutput,
int replications,
int bandits,
int maxSteps,
BanditFactory modelFactory,
DistributionGenerator refSampler)
Computes average regret relative to perfect knowledge given uniform random probabilities.

public static void main(String[] args) throws FileNotFoundException, NoSuchMethodException, InvocationTargetException, InstantiationException, IllegalAccessException, InterruptedException
public static double commitTime(String outputFile, int n, double p1, double p2, int cutoff) throws FileNotFoundException
plot(tapply(z$k, floor(z$i/10), mean), type='l')
outputFile
 Where to write resultsn
 How many steps to followp1
 First probability of rewardp2
 Second probability of rewardcutoff
 Only keep results after this many stepsFileNotFoundException
 If the directory holding the output directory doesn't exist.public static double averageRegret(String outputFile, int[] sizes, int replications, int bandits) throws FileNotFoundException
This can be plotted pretty much directly in R
> x=read.delim(file='~/Apache/stormaggregator/regret.tsv') > bxp(list(com.mapr.stats=t(as.matrix(x[,2:6])), n=rep(1000,times=8),names=x$n))
outputFile
 Where to put the outputsizes
 The different size experiments to usereplications
 Number of times to repeat the experimentbandits
 How many bandits to simulateFileNotFoundException
 If the output file can't be opened due to a missing directory.public static double totalRegret(String cumulativeOutput, String perTurnOutput, int replications, int bandits, int maxSteps, BanditFactory modelFactory, DistributionGenerator refSampler) throws FileNotFoundException
This can be plotted pretty much directly in R
> x=read.delim(file='~/Apache/stormaggregator/regret.tsv') > bxp(list(com.mapr.stats=t(as.matrix(x[,2:6])), n=rep(1000,times=8),names=x$n))
cumulativeOutput
 Where to write the cumulative regret resultsperTurnOutput
 Where to write the per step regret resultsreplications
 How many times to replicate the experimentbandits
 How many bandits to emulatemaxSteps
 Maximum number of trials to run per experimentmodelFactory
 How to construct the solver.refSampler
 How to get reward distributions for banditsFileNotFoundException
 If the output file can't be opened due to
a missing directory.Copyright © 2013. All Rights Reserved.