Logged user interactions are one of the most ubiquitous forms of data available because they can be recorded from a variety of systems (e.g., search engines, recommender systems, ad placement) at little cost. Naively using this data, however, is prone to failure. A key problem lies in biases that systems inject into the logs by influencing where we will receive feedback (e.g., more clicks at the top of the search ranking). This talk explores how counterfactual inference techniques can make learning algorithms robust against bias. This makes log data accessible to a broad range of learning algorithms, from ranking SVMs to deep networks.