Asset returns are frequently assumed to be determined by one or more common factors. We consider a bivariate factor model, where the unobservable common factor and idiosyncratic errors are stationary and serially uncorrelated, but have strong dependence in higher moments. Stochastic volatility models for the latent variables are employed, in view of their direct application to asset pricing models. Assuming the underlying persistence is higher in the factor than in the errors, a fractional cointegrating relationship can be recovered by suitable transformation of the data. We propose a narrow band semiparametric estimate of the factor loadings, which is shown to be consistent with a rate of convergence, and its finite sample properties are investigated in a Monte Carlo experiment.