Drift-Aware Multi-Pass Streaming Training for Generative Recommendation on Non-Stationary User Sequences

Aowei Shen; Minjae Rhee; Changhao Zhang; Yinuo Wang

doi:10.54691/fpxe6745

Authors

Aowei Shen University of Washington, Seattle, USA
Minjae Rhee University of Illinois Urbana-Champaign, Champaign, USA
Changhao Zhang University of Chicago, Chicago, USA
Yinuo Wang University of Chicago, Chicago, USA

DOI:

https://doi.org/10.54691/fpxe6745

Keywords:

Generative recommendation, semantic IDs, streaming training, concept drift, continual learning, recommender systems.

Abstract

Generative recommenders that represent an item as a short tuple of semantic tokens and decode the next item autoregressively have become a competitive alternative to embedding-based retrieval, but they are almost always studied on a static snapshot of the data. Deployed streams are not static: catalogues rotate, popularity bursts and user intent shifts, so the recommender must be re-trained repeatedly as blocks of interactions arrive. This paper asks how that re-training should be scheduled. We first run a block-level probe study on two real streams, forking the model at every update cycle and measuring how next-block accuracy responds to the number of optimisation passes and to the fraction of replayed history. The two streams disagree: on a fast-rotating stream (21% of the items in an average block are new) replaying half of the batch from memory costs 28% Recall@10 and three passes are enough, whereas on a slow stream replay helps and five passes still improve accuracy. No fixed schedule is therefore correct across streams, and per-stream tuning is impossible online. We propose DAMP, which estimates drift online from the divergence of the incoming block combined with the model's excess loss on that block, and uses a single drift score to gate four coupled decisions: the number of passes, the replay ratio, the distillation strength, and which memory samples are still relevant. On two real streams, DAMP is statistically tied with the strongest fixed schedule on the volatile one (p=0.772) while beating every stability-oriented baseline there, and on the stable one it is significantly better than all eight baselines (18.6% over fixed multi-pass training, p<0.001), including at matched compute. No fixed schedule is strong on both. We also report where the controller fails: on a synthetic generator whose item-frequency divergence is large while its preference structure stays partly stationary, our absolute drift calibration closes the replay gate when it should not. Code, data pipeline and raw logs are released.

Downloads

Download data is not yet available.

References

[1] Hidasi, B., Karatzoglou, A., Baltrunas, L., & Tikk, D. (2016). Session-based recommendations with recurrent neural networks. In Proceedings of the International Conference on Learning Representations.

[2] Kang, W.-C., & McAuley, J. (2018). Self-attentive sequential recommendation. In Proceedings of the IEEE International Conference on Data Mining (pp. 197–206).

[3] Sun, F., Liu, J., Wu, J., Pei, C., Lin, X., Ou, W., & Jiang, P. (2019). BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (pp. 1441–1450).

[4] Rajput, S., Mehta, N., Singh, A., Keshavan, R. H., Vu, T., Heldt, L., Hong, L., Tay, Y., Tran, V. Q., Samost, J., Kula, M., Chi, E. H., & Sathiamoorthy, M. (2023). Recommender systems with generative retrieval. In Advances in Neural Information Processing Systems (Vol. 36).

[5] Tay, Y., Tran, V. Q., Dehghani, M., Ni, J., Bahri, D., Mehta, H., Qin, Z., Hui, K., Zhao, Z., Gupta, J., Schuster, T., Cohen, W. W., & Metzler, D. (2022). Transformer memory as a differentiable search index. In Advances in Neural Information Processing Systems (Vol. 35).

[6] Geng, S., Liu, S., Fu, Z., Ge, Y., & Zhang, Y. (2022). Recommendation as language processing (RLP): A unified pretrain, personalized prompt & predict paradigm (P5). In Proceedings of the 16th ACM Conference on Recommender Systems.

[7] Zhai, J., Liao, L., Liu, X., Wang, Y., Li, R., Cao, X., Gao, L., Gong, Z., Gu, F., He, J., Lu, Y., & Shi, Y. (2024). Actions speak louder than words: Trillion-parameter sequential transducers for generative recommendations. In Proceedings of the 41st International Conference on Machine Learning (Vol. 235, pp. 58484–58509). PMLR.

[8] Zheng, B., Hou, Y., Lu, H., Chen, Y., Zhao, W. X., Chen, M., & Wen, J.-R. (2024). Adapting large language models by integrating collaborative semantics for recommendation. In Proceedings of the IEEE 40th International Conference on Data Engineering (pp. 1435–1448).

[9] Singh, A., Vu, T., Mehta, N., Keshavan, R., Sathiamoorthy, M., Zheng, Y., Hong, L., Heldt, L., Wei, L., Tandon, D., Chi, E. H., & Yi, X. (2024). Better generalization with semantic IDs: A case study in ranking for recommendations. In Proceedings of the 18th ACM Conference on Recommender Systems (pp. 1039–1044).

[10] Dooms, S., De Pessemier, T., & Martens, L. (2013). MovieTweetings: A movie rating dataset collected from Twitter. In Workshop on Crowdsourcing and Human Computation for Recommender Systems (CrowdRec at RecSys).

[11] Shen, Z., Zhao, W., Wang, B., Wang, Z., & Shang, W. (2026). CAGR: A cross-accelerator graph optimization framework for efficient recommender system inference. IEEE Access, 14, 38544–38562.

[12] Mi, F., Lin, X., & Faltings, B. (2020). ADER: Adaptively distilled exemplar replay towards continual learning for session-based recommendation. In Proceedings of the 14th ACM Conference on Recommender Systems (pp. 408–413).

[13] Mi, F., & Faltings, B. (2020). Memory augmented neural model for incremental session-based recommendation. In Proceedings of the 29th International Joint Conference on Artificial Intelligence (pp. 2169–2176).

[14] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., Hassabis, D., Clopath, C., Kumaran, D., & Hadsell, R. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13), 3521–3526.

[15] Li, Z., & Hoiem, D. (2018). Learning without forgetting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 40(12), 2935–2947.

[16] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017). iCaRL: Incremental classifier and representation learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

[17] Ash, J. T., & Adams, R. P. (2020). On warm-starting neural network training. In Advances in Neural Information Processing Systems (Vol. 33, pp. 3884–3894).

[18] van den Oord, A., Vinyals, O., & Kavukcuoglu, K. (2017). Neural discrete representation learning. In Advances in Neural Information Processing Systems (Vol. 30).

[19] Lee, D., Kim, C., Kim, S., Cho, M., & Han, W.-S. (2022). Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 11513–11522).

[20] Hou, Y., He, Z., McAuley, J., & Zhao, W. X. (2023). Learning vector-quantized item representation for transferable sequential recommenders. In Proceedings of the ACM Web Conference.

[21] Hou, Y., Mu, S., Zhao, W. X., Li, Y., Ding, B., & Wen, J.-R. (2022). Towards universal sequence representation learning for recommender systems. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining.

[22] Petrov, A., & Macdonald, C. (2024). RecJPQ: Training large-catalogue sequential recommenders. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining (pp. 538–547).

[23] He, X., Zhang, H., Kan, M.-Y., & Chua, T.-S. (2016). Fast matrix factorization for online recommendation with implicit feedback. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval.

[24] Wang, W., Yin, H., Huang, Z., Wang, Q., Du, X., & Nguyen, Q. V. H. (2018). Streaming ranking based recommender systems. In Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval.

[25] Vitter, J. S. (1985). Random sampling with a reservoir. ACM Transactions on Mathematical Software, 11(1), 37–57.

[26] Guo, L., Yin, H., Wang, Q., Chen, T., Zhou, A., & Nguyen, Q. V. H. (2019). Streaming session-based recommendation. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 1569–1577).

[27] Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the knowledge in a neural network. arXiv:1503.02531.

[28] Chaudhry, A., Rohrbach, M., Elhoseiny, M., Ajanthan, T., Dokania, P. K., Torr, P. H. S., & Ranzato, M. (2019). On tiny episodic memories in continual learning. arXiv:1902.10486.

[29] Lopez-Paz, D., & Ranzato, M. (2017). Gradient episodic memory for continual learning. In Advances in Neural Information Processing Systems (Vol. 30, pp. 6467–6476).

[30] Buzzega, P., Boschini, M., Porrello, A., Abati, D., & Calderara, S. (2020). Dark experience for general continual learning: A strong, simple baseline. In Advances in Neural Information Processing Systems (Vol. 33).

[31] Fan, H., Jiao, Y., Wang, M., Chen, J., & Yang, S. (2026). Fraud learns too: Continual graph learning under strategic adversarial drift in dynamic networks. Scientific Reports. https://doi.org/10.1038/s41598-026-60997-7

[32] Mo, T., Zhang, C., Zou, J., Guo, Z., & Rhee, M. (2026). Self-evolving AI agents with dual memory for automated software testing and bug localization. IEEE Access, 14, 111086–111102.

[33] Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014). A survey on concept drift adaptation. ACM Computing Surveys, 46(4), Article 44.

[34] Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., & Zhang, G. (2019). Learning under concept drift: A review. IEEE Transactions on Knowledge and Data Engineering, 31(12), 2346–2363.

[35] Bifet, A., & Gavaldà, R. (2007). Learning from time-changing data with adaptive windowing. In Proceedings of the SIAM International Conference on Data Mining.

[36] Rhee, M., Zou, J., Mo, T., Teng, D., & Yang, J.-S. (2026). Enhancing web search agents with self-play contrastive fine-tuning. IEEE Access. https://doi.org/10.1109/ACCESS.2026.3717594

[37] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. In Advances in Neural Information Processing Systems (Vol. 30).

[38] Harper, F. M., & Konstan, J. A. (2015). The MovieLens datasets: History and context. ACM Transactions on Interactive Intelligent Systems, 5(4), Article 19.

[39] Loshchilov, I., & Hutter, F. (2019). Decoupled weight decay regularization. In Proceedings of the International Conference on Learning Representations.