Pavel Vasilev | Ural Federal University

About Me

I am a student at Yandex School of Data Analysis (YSDA) (2024–2026) and Ural Federal University (UrFU) (2022–2026), majoring in Fundamental Informatics and Information Technologies.

My primary research interests lie in Machine Learning, specifically Deep Learning, Computer Vision, and NLP. I am currently focused on Mixture of Experts (MoE) architectures, Multimodal Models, and their interpretability.

Schools & Workshops:

Federated Learning School (MBZUAI × YSDA): Studied Mixture of Experts (MoE), focusing on the trade-off between objective and balance functions, and analyzing various routing mechanisms. Mentors: Martin Takáč and Aleksandr Beznosikov.
Vega Summer School on Financial Mathematics (Kaluga, Summer 2025): Participated in an intensive school organized by Vega Institute, covering topics in advanced financial mathematics and stochastic analysis.

News

[Aug. 2025 - Present] Working as a Middle DL Engineer at Sberbank (Gigacode team), developing an AI coding assistant.
[Apr. 2025 - July 2025] Interned as an NLP Engineer at Yandex (Alice team), working on function calling functionality.
[July 2024 - Sept. 2024] Interned as a C# Backend Engineer at SKB Kontur, working on internal Kubernetes-like systems.
[2024] Joined Yandex School of Data Analysis.

Research & Projects

Research: Expert Specialization in Multimodal Diffusion Models

Pavel Vasilev, Daniil Tikhonov. Mentor: Gleb Molodtsov.

An in-depth investigation into replacing Feed-Forward Networks (FFN) with Mixture of Experts (MoE) in unified multimodal transformer architectures (like Show-o). We explored how models can autonomously specialize experts for different data modalities (Text vs. Image) and domains. Read more details…

Attention Sinks as a Signal: A Systematic Empirical Study

Pavel Vasilev, Shevchenko Artem. Affiliation: HSE, YSDA.

We study sink mass—attention on the first few tokens—as a predictor of correctness on TruthfulQA across multiple models. We find strong heterogeneity in the signal direction and modest predictive quality (AUROC ~0.44–0.61), showing that sink mass does not consistently add value beyond next-token entropy. Read more details…

InferD: Distributed LLM Inference Engine

Engineering Project

Developed a decentralized engine for LLM inference across multiple nodes. Implemented load balancing using distributed hash tables (DHT), inspired by SWARM and Petals. Read more details…