This paper proposes several statistical tests for finite state Markov games to examine the null hypothesis that data from distinct markets can be pooled. We formulate tests of (i) the conditional choice and state transition probabilities, (ii) the steady-state distribution, and (iii) the conditional state distribution given an initial state. If the null cannot be rejected, then the data across markets can be pooled. A rejection of the null implies that the data cannot be pooled across markets. In a Monte Carlo study we find that the test based on the steady-state distribution performs well and has high power even with small numbers of markets and time periods. We apply the tests to the empirical study of Ryan (2012) that analyzes dynamics of the U.S. Portland Cement industry and assess if the single equilibrium assumption is supported by the data.