Speaker Labeling & Call for Community Contribution Workflow

[MrH2T]

After several days of intensive development, I have completed a preliminary speaker labeling for the game strings. The current scripts and raw data are available in the output/results/ directory (refer to all_chapters_speaker_results.csv and all_chapters_dump_aligned_results.csv).

Currently, the automated labeling accuracy maintains a baseline of 50%–60%. To further refine this dataset, I wish we can implement a community-driven maintenance workflow. This would allow contributors to verify existing labels, correct misidentifications, or submit their own labeled datasets to improve the overall model precision.

https://drive.google.com/file/d/1WZeFz4ouzA3zf1D3Y4TKI8ni1AziPaKE/view?usp=drive_link

(I don't know why I cannot attach files)

[MrH2T]

FYI It's an automated inference coded with GPT and tested on wiki corrected strings. That's why I think there should be an online check-and-justify ui

[HushBugger]

Thank you for this effort, it sounds interesting! (The Drive link is not shared publicly so I can't look at it yet, I requested access.)

Unfortunately I suspect this isn't the best way to go about it... I've done preliminary labeling for chapter 1 and 2 in #4 (comment) and it seemed to me that the best way to do it was for me to use source code heuristics to label many lines automatically (the cutscene system helps, e.g. https://code.deltarune.wiki/ch3/gml_object_obj_ch3_gsa02_step_0#L309 (if so far you have been labeling using just the text then I recommend adding the source code context as data)) and manually go through everything to check the automatic output and label the rest by referencing source code and longplays.

This takes significant work but it's not prohibitive, I think I did a chapter a day. Crowdsourcing corrections after the end would be nice but the initial work can be done by one person.

There are also decisions to be made about how to label e.g.:

• Text with multiple speakers
• Text with dialogue choices (people who believe Kris decides these would love being able to filter for this I'm sure)
• Different kinds of non-character text, narrator vs menu vs item description
• Item descriptions that are written by Lancer or Rouxls Kaard
• Unused text
• Duplicated text from previous chapters

I feel these decisions are best made and enforced by someone who's deeply immersed in the data by doing the manual process above. (Though ideally in a way that permits easy relabeling later.)

A final problem is that even if I had labels I wouldn't want to display them until I optimize the page with lazy loading, it's already too heavy as is. But I've been working hard on that and hope to finish it soonish.

I do know that a labeled dump is a resource many people have been looking for. Doing everything exactly the way I want it is going to take a while, so if in the meantime you put a spreadsheet up they might be interested.

[MrH2T]

My current approach involves using a small interpreter to infer context from the source code, including speaker names, and inferring things similar to UTMT from various strings (e.g., SPR names), and so on.

[HushBugger]

Nice! That sounds more sophisticated than what I did.

I assumed you intended this as a contribution to the text dump but it might make more sense as its own project?

[MrH2T]

I thought it'd be cool to add this to that already famous text dump — just for your reference though :)

[HushBugger]

Thank you, I'm flattered! I really do encourage you to give it its own home though. I originally made the UT dialogue dump because I stumbled across maxdefolsch's spreadsheet with all the lines, it's good to have data out in the open.