This lecture continues to discuss issues related to estimating absolute performance from transient and steady-state simulations (of terminating and non-terminating systems, respectively). We continue to emphasize the importance and utility of interval estimations (over point estimates). We then move on to discuss experimental methodologies useful for steady-state simulations, particularly related to eliminating estimator bias and reducing computational time. [ due to an instructor error, the lecture from 2021-11-09 was not recorded, and the archived 2020-11-010lecture is re-used here instead ]

In this lecture, we continue to introduce terminating and non-terminating systems and difference methods for estimating performance from simulation models of them (using transient and steady-state simulations). This involves a description of various types of point estimators (mean and quantile) as well as related interval estimators (confidence intervals and prediction intervals, as well as the relationship to standard error of the mean (SEM)). We start to discuss issues involving making inferences from pseudo-replicated within-replication samples versus across-replication samples (which are independent and often normally distributed). We will continue this in the next lecture, as we start focusing more on steady-state simulations of non-terminating systems. [ For some strange reason, the in-room video camera was not recorded as the speaker view despite apparently working during the class. Consequently, only the slide view is shown. ]

In this lecture, we review the fundamental tradeoffs in hypothesis testing and the concrete origins of the assumptions in both the t-test and Chi-square test. We also discuss parametric and non-parametric statistics (including exact and non-exact tests) and how non-parametric, exact statistics like the Kolmogorov–Smirnov test are derived. This culminates in a discussion of the multiple comparisons (MC) problem and the Bonferroni correction as well as alternative tests (such as a MANOVA or an ANOVA with post hoc test) that have more statistical power than the Bonferroni correction. We close with an introduction to performance inference from simulation, which we will continue discussing in the next 3 lectures.

In this halloween-themed lecture, we go into more detail on the foundations of hypothesis testing – specifically hypothesis testing with small sample sizes. This allows us to talk about where the Student's t test comes from (and why it is defined that way) as well as where the Chi-square test comes from (and why it is defined that way). Throughout the lecture, we highlight the importance of statistical power and do a power analysis example for a paired-difference t-test.

In this lecture, we review summary statistics, MLE, and goodness-of-fit tests (particularly Chi-square and Kolmogorov–Smirnov, with some mention of Anderson–Darling and Shapiro–Wilk), with a particular focus on the type-I error, type-II error, and statistical power. We then introduce verification, validation, and calibration of simulation models and close with an example for the simulation of a bank. We use rigorous statistical methods to drive the calibration process that leads to updating the model of the bank and ensuring its outputs are a good statistical match for outputs in a real bank. This involves making use of a power analysis for a one-sample, two-sided t-test. We will cover the paired t-test version of this problem in the next lecture.

In this lecture, we start out with Q-Q and P-P probability plots that we did not have time to cover from last time. We then transition to a review about type-I error and p values and try to motivate the topics of STATISTICAL POWER and EFFECT SIZES, which we will dive into more in the next few lectures. We then discuss summary statistics and how to use methods such as maximum likelihood estimation (MLE) to come up with good choices of parameters for distributions picked in the input modeling process. Next time, we will discuss testing the (goodness of) fit for those parameterized distributions.

In this lecture, we continue our discussion of input modeling in depth. We start with a more detailed example of how data collection can guide the choice of the structural features of a system. We then move to the point in the process when the structure of the model is set but the input models have to be chosen based on collected data. We cover methods for generating histograms and matching those histograms to common distributions (both discrete and continuous). We stop just before discussing Q-Q plots and P-P plots, which we will pick up next time along with discussing how to parameterize these chosen distributions.

In this lecture, we review topics from the first half of the semester that will be tested over in the upcoming midterm. Most of the class involves working examples on the whiteboard.  Whiteboard notes captured for this session can be found at: https://www.dropbox.com/s/pih0wt3abwbatbb/IEE475-LectureF-2021-09-30-Midterm_Review-Whiteboard_Notes.pdf?dl=0

In this lecture, we finish covering tests of uniformity (Chi-squared and Kolmogorov–Smirnov) and independence (autocorrelation and runs (above and below) tests) for pseudo-random number generators (PRNGs). We then move on to discussing the details of inverse-transform sampling for random-variate generation. We cover how to derive a CDF from a piecewise PDF and how to invert a CDF to produce a quantile function fit for random-variate generation. We also discuss the discrete inverse-transform case.

We start the lecture covering some discrete random variables that we did not get to during Lecture D2. We also introduce the Poisson process and how it relates to the Poisson and exponential random variables. We then pivot to discussing pseudo-random number generators (PRNGs), including their required as well as desired properties and statistical tests to test for independence and uniformity. We will continue the discussion of statistical tests for independence at the start of next lecture (Lecture E2).