Götz-Henrik Wiegand

Götz-Henrik Wiegand

he/him

PhD Student in Natural Language Processing @ University St. Gallen (HSG) and Uni-Konstanz

Publications

Conference Papers

Is More Data Worth the Cost? Dataset Scaling Laws in a Tiny Attention-Only Decoder

Published in IEEE SDS 2026 (Full Paper, Zürich) & ICLR 2026 Workshop on Data-FM (Rio de Janeiro, Brazil), 2026

Shows that dataset scaling laws hold even at small scales, where training on only 30% of data achieves about 90% of full performance in a tiny attention-only decoder.

Recommended citation: Wiegand, G.-H., Raichle, L., Staedeli, R., Handschuh, S., Hrycej, T., & Bermeitinger, B. (2026). "Is More Data Worth the Cost? Dataset Scaling Laws in a Tiny Attention-Only Decoder." IEEE International Conference on Data Science (SDS 2026), Zürich, Switzerland.

A Convexity-Dependent Two-Phase Training Algorithm for Deep Neural Networks

Published in Proceedings of the 17th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K KDIR 2025), 2025

Proposes a two-phase optimization algorithm that detects when the loss landscape transitions from non-convex to convex regions, switching from Adam to Conjugate Gradient to substantially improve convergence speed and accuracy.

Recommended citation: Hrycej, T., Bermeitinger, B., Pavone, M., Wiegand, G.-H., & Handschuh, S. (2025). "A Convexity-Dependent Two-Phase Training Algorithm for Deep Neural Networks." Proceedings of the 17th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management, 78–86.
Download Paper

Leveraging Generative AI for Enhancing Domain-Driven Software Design

Published in Proceedings of the Upper-Rhine Artificial Intelligence Symposium (URAI 2024), 2024

Demonstrates that a 4-bit quantized Code Llama fine-tuned with LoRA on a consumer GPU can generate syntactically correct domain-specific JSON objects for Domain-Driven Design, partially automating metamodel creation.

Recommended citation: Wiegand, G.-H., Stepniak, F., & Baier, P. (2024). "Leveraging Generative AI for Enhancing Domain-Driven Software Design." Proceedings of the Upper-Rhine Artificial Intelligence Symposium, 2024, 41–50.
Download Paper