We introduce the concept of pattern graphs--directed acyclic graphs representing how response patterns are associated. A pattern graph represents an identifying restriction that is nonparametrically identified/saturated and is often a missing not at random restriction. We introduce a selection model and a pattern mixture model formulations using the pattern graphs and show that they are equivalent. A pattern graph leads to an inverse probability weighting estimator as well as an imputation-based estimator. Asymptotic theories of the estimators are studied and we provide a graph-based recursive procedure for computing both estimators. We propose three graph-based sensitivity analyses and study the equivalence class of pattern graphs.