Human-Computer Interaction
See recent articles
Showing new listings for Friday, 31 July 2026
- [1] arXiv:2607.27212 [pdf, html, other]
-
Title: Digital Harf: A Clinically Integrated Multimodal AI System for Pervasive Arabic Speech and Language TherapyAsif Azad, Mohammad Sadat Hossain, MD Sadik Hossain Shanto, Sabri Boughorbel, Abdulrhman Aljouie, Bdour Alwuqaysi, Yahya Bokhari, Ayah Othman Sindi, Ehsan HoqueSubjects: Human-Computer Interaction (cs.HC); Computation and Language (cs.CL)
Children with Autism Spectrum Disorder in Arabic-speaking countries face compounded barriers to effective speech and language therapy: a shortage of qualified specialists, limited service reach beyond urban centers, and a near-total absence of culturally grounded digital therapy materials. We present Digital Harf, a pervasive, multimodal AI platform that extends clinician-led speech and language therapy into the home. The system integrates three therapeutic modules - language therapy, speech intelligibility, and picture description - within a unified workflow that adapts to each child's performance over time. To address the critical shortage of Arabic therapy content, we introduce an Agentic Synthetic Data Engine (ASDE) that automatically generates culturally relevant images, prompts, and language tasks guided by explicit therapeutic and cultural criteria. Expert evaluation with 13 licensed Speech-Language Pathologists yielded a 90.1% clinical acceptance rate for ASDE-generated content without any manual curation or selection, and strong ratings for cultural and linguistic alignment across the full platform. Digital Harf demonstrates that AI-driven therapeutic systems can be built from the ground up for underrepresented linguistic settings, treating cultural grounding as core infrastructure rather than an adaptation afterthought.
- [2] arXiv:2607.27369 [pdf, html, other]
-
Title: Evaluating the Vergence-Accommodation Conflict in Gaze-Based 3D Target SelectionMohammad Raihanul Bashar, Mohammadreza Amini, Aunnoy K Mutasim, Mayra Donaji Barrera Machuca, Wolfgang Stuerzlinger, Anil Ufuk BatmazComments: 9 Pages, 7 Figures, IEEE ISMAR 2026 (TVCG)Journal-ref: IEEE Transactions on Visualization and Computer Graphics (TVCG) Special Issue on IEEE ISMAR 2026Subjects: Human-Computer Interaction (cs.HC)
State-of-the-art head-mounted displays (HMDs) enable gaze-based selection in virtual environments. Yet, these HMDs suffer from the vergence-accommodation conflict (VAC), which is known to affect interaction performance. The VAC might influence gaze-based selection performance because it directly affects eye-movement behavior. Thus, in this paper, we investigate how the VAC influences gaze-based 3D target selection across varying depth conditions. Our results show that as the (visual) depth increases, user performance significantly decreases with gaze-based selection. Moreover, a previously suggested Variation in Diopter Fitts' law model captured this performance change better relative to a linear model. These findings provide evidence that gaze-based pointing is negatively affected by the VAC and highlight the importance of accounting for depth-dependent factors when designing gaze-based interaction in 3D environments.
- [3] arXiv:2607.27697 [pdf, html, other]
-
Title: DP-LENS: A Density-Aware Polyfocal Lens with Topology-Driven Auto-Routing for Occlusion Management in Immersive 3D AnalyticsComments: Accepted to IEEE International Symposium on Mixed and Augmented Reality (ISMAR) 2026, to appear in IEEE Transactions on Visualization and Computer Graphics (TVCG). 11 pagesSubjects: Human-Computer Interaction (cs.HC)
Immersive environments, e.g., virtual reality (VR), offer a unique approach to exploring complex 3D datasets, where data is often heavily occluded and exploration incurs a high cognitive load. We propose DP-LENS, a density-aware polyfocal fisheye lens equipped with topology-driven auto-routing. While preserving peripheral context through geometric deformation and 3D perspective techniques, it enables users to explore 3D data with a lower cognitive load. To facilitate hands-free macro-navigation, we integrate a Large Language Model (LLM) to serve as a supplementary voice-based target selection tool that initiates the auto-routing algorithm. Two user studies with 34 participants investigate the potential benefits of this system. Our first study (N=18) compared the manual DP-LENS against two industry-standard baselines (i.e., World-in-Miniature and volumetric slicing) in heavily occluded 3D datasets. The results show that DP-LENS significantly reduced cognitive load, decreased completion time, and improved user preference. The second study (N=16) compared the topology-driven auto-routing system (initiated via voice commands) with a fully manual DP-LENS. The results show that the auto-routing system improved task efficiency, further reduced cognitive load, and garnered higher user preference. Furthermore, the auto-routing partially decoupled exploration efficiency from the physical dimensions of the data and mitigated physical fatigue to some extent. Based on the findings, we proposed design implications to inform the development of more spatially scalable and low-fatigue interactions for future 3D visual analytics systems.
- [4] arXiv:2607.27863 [pdf, html, other]
-
Title: Creative Task Cards for Reflection, Self-Efficiency and Self-Regulation in CS1 Introductory Programming: Initial InsightsComments: In Proceedings of The First Reflection in Creative Experience (RiCE) Workshop (RiCE W1) arXiv:2607.24558Subjects: Human-Computer Interaction (cs.HC)
Computer Science students in introductory programming courses struggle with emotional challenges such as low levels of sustained interest and a lack of self-belief. However, students are typically introduced to disciplinary strategies (such as debugging strategies) and not the metacognitive nor self-regulation strategies that could help them overcome such emotional challenges. This paper addresses this issue by introducing creative task cards designed to support students' reflection, self-efficacy, and self-regulation in programming assignments. To evaluate the cards, twenty-nine students used them to complete a mock coding assignment in-class and participated in surveys and focus groups. Our initial qualitative insights suggest that students: found value in taking breaks, particularly when they could leave the classroom to reflect with peers; could use drawing to better reflect on feelings of cognitive overload; and felt more capable progressing with their work once breaking it into smaller steps.
- [5] arXiv:2607.27938 [pdf, html, other]
-
Title: VizPilot: Automated Onboarding for SVG-based Composite Visualizations using Multimodal LLMsComments: Accepted by IEEE VIS 2026Subjects: Human-Computer Interaction (cs.HC)
Composite visualizations integrate multiple visualizations to represent complex datasets effectively, but their intrinsic composite designs often impose a high initial cognitive load on novice users. Existing visualization onboarding approaches are typically platform-dependent, require substantial manual authoring effort, and struggle with the structural complexity of composite visualizations, limiting their general applicability. We present VizPilot, an automated visualization onboarding approach that reverse-engineers composite visualization structure to generate interactive onboarding experiences directly from raw visualization artifacts. VizPilot consists of two modules: a Composite Visualization Analyzer and an Onboarding Interface. Leveraging Multimodal Large Language Models (MLLMs), the Analyzer employs a two-stage pipeline that decomposes a visualization into visual components, extracts structured explanations, and maps them to precise SVG elements for reliable highlighting and interaction. Implemented as a browser extension, VizPilot requires only a brief visualization description and optional interaction source code from the visualization developer to automatically generate onboarding content. The Onboarding Interface supports both guided narrative scrollytelling and free exploration, enabling users to learn visualization components progressively or on demand. We evaluate VizPilot through a comparative analysis of different input modalities, a usage scenario demonstrating reduced authoring effort, and a user study assessing its impact on users' cognitive load. The results demonstrate that VizPilot effectively automates the authoring of onboarding experiences while improving the usability and accessibility of composite visualizations.
- [6] arXiv:2607.28072 [pdf, html, other]
-
Title: Investigating Effective Uncertainty Visualizations for Ordinal Crowdsourced Data of Crowding ConditionsBea Alexis Arcega, Annika Dominique S. Campos, Kathleen Therese Cruz, Aaron Ace Toledo, Briane Paul V. SamsonSubjects: Human-Computer Interaction (cs.HC)
Commuters often encounter crowding in railway systems, particularly in queues where passenger density varies throughout the day. This introduces uncertainty in crowdedness, making it difficult for individuals to anticipate conditions and plan their trips effectively. Crowdsourcing has been a valuable method for collecting localized user data. But the unpredictability of crowds and the uncertainty of crowdsourced information pose new challenges for decision-making. However, we know little about how to effectively visualize uncertainty in crowdedness to support informed commuting decisions, particularly when using crowdsourced ordinal data. Here, we investigated different uncertainty visualizations and their effectiveness in representing the variability and reliability of crowdsourced crowding data. They were evaluated through an online study, and we found that cluster visualization is best suited to reduce cognitive load while maximizing user confidence and trust. On the other hand, participants showed higher accuracy in determining crowd levels when using bubble treemaps.
- [7] arXiv:2607.28155 [pdf, other]
-
Title: A Mathematical Framework for Reading the Autopsias' Meta - Compositional SystemComments: 13 pages, 12 figures, 11 equationsSubjects: Human-Computer Interaction (cs.HC)
Background. New forms of music writing using computers have arisen in the past 20 years. Most of them use the capacities of digital manipulation of data like animation, algorithmic processing, cinematic view, and much more. These scores use dynamic musicography, and all of them share a problem. They have readability problems. We will argue that this problem can be addressed by mathematical tools. Aims. Take the Autopsias [Autopsies] meta-compositional system as a study case for starting the construction of a mathematical framework that can overview the readability for cynetic musicography. The Autopsias system is the process of transforming a musical score dynamically. Methodology. We will start to build a mathematical framework by taking a group of basic topological concepts, and applying them after a bridge between a printed score and a dynamic computational re-appropriation of it has taken place. We will observe the writing and the performing of the several performances of the Autopsias composition, and analyze them from a mathematical perspective, in order to get the ability to start constructing a new set of orthographical tools for the performance of dynamic musicography. Main Contribution. The performances of Autopsias show that mathematics, as with pitch-class sets, can be helpful for understanding music information, as well as its performance. Also show the necessity of augmenting the set of orthographic rules, in order to improve the dialog between composers and performers.
- [8] arXiv:2607.28203 [pdf, html, other]
-
Title: Student Perceptions and Preferences Regarding AI-Generated Instructional Videos in Computing EducationSubjects: Human-Computer Interaction (cs.HC)
Students differ in how they prefer to engage with learning resources, with some favoring textual materials and others visual or video-based content. Recent advances in generative AI have led CS education research to focus on text-based AI tools for developing learning resources. However, advances in AI video models and the rapid proliferation of AI video generation tools have made it possible for instructors to create high-quality personalized educational videos efficiently and cost-effectively. Understanding students' perceptions of AI-generated videos is thus critical for helping CS instructors know when and how to use them purposefully. To address this gap, we conducted a descriptive post-test survey study in which 170 computing students at two U.S. institutions watched three 3-minute AI videos on the Markdown markup language created with Knowlify. Students then completed a survey about their perceptions of the Markdown videos and their broader views on the use of AI-generated videos in education. Students rated the Markdown videos as high-quality, accurate, and usable, with nearly half unable to determine whether the videos were AI-generated. At the same time, students expressed limited comfort with the widespread adoption of AI videos in the classroom. They preferred AI videos for simple, supplemental, and visual use cases, while expressing concerns about lower-quality or inaccurate content, reduced instructor interaction, and diminished educational value.
- [9] arXiv:2607.28204 [pdf, html, other]
-
Title: Toward Annotation-Efficient Continuous Emotion Arousal Quantification via Group-Level EEG Dynamic Neural SynchronySubjects: Human-Computer Interaction (cs.HC)
Continuous emotional arousal quantification remains bottlenecked by time-consuming and labor-intensive manual annotation. This work investigates group-level EEG dynamic neural synchrony (DNS) as a principled signal for continuous arousal quantification that bypasses per-subject manual labeling. Using Correlated Component Analysis (CorrCA) with sliding-window computation across four EEG datasets spanning 142 subjects and over 207 hours, we systematically evaluate DNS as a group-level marker for emotional arousal dynamics. Three key findings emerge. First, DNS exhibits significant emotion information from valence-dependent differences (all p<0.003), with positive emotions eliciting higher synchrony. Second, DNS correlates more strongly with the first-order derivative of arousal than with raw arousal values, revealing that neural synchrony captures the rate of emotional change rather than static intensity. Third, we provide the first systematic characterization of how DNS-arousal coupling depends on key methodological choices, finding that moderate windows (10-30 s), positive lags (0-10 steps), and First-order Difference feature of EEG from the dominant CorrCA component yield consistently strong coupling. Subject-split replication and block permutation tests confirm these associations are not statistical artifacts. Our findings establish DNS as an empirically validated group-level marker toward annotation-efficient continuous emotional arousal quantification.
- [10] arXiv:2607.28239 [pdf, other]
-
Title: Identifying a Level-up Pathway for AI-assisted Counterspeech through ElaborationSubjects: Human-Computer Interaction (cs.HC)
Given the profound societal impact of vaccine-skeptical content on social media, community-driven counterspeech has emerged as a promising participatory response to contest and curb such objectionable content. Yet crafting effective counterspeech remains challenging for ordinary users, limiting their willingness and ability to engage constructively. We designed and evaluated three generative AI-assisted counterspeech writing systems that vary by assistance stage (co-writing vs. re-writing) and mode (guided vs. unguided) to support lay users' responses to vaccine-skeptical content. We ask whether AI can help users craft counterspeech perceived as both effective and authentic, which forms of AI support work best, and through what mechanisms. In a randomized controlled trial with social media users, participants wrote counterspeech responses to both statistical and narrative vaccine-skeptical content. Across evidence types, AI-assisted writing increased perceived counterspeech effectiveness while largely preserving authentic self-expression, and perceived effectiveness was the strongest predictor of willingness to counterspeak publicly. AI's primary benefit was facilitating more elaborate writing, producing messages that were more informative, analytical, and lexically sophisticated. These findings suggest a level-up pathway for AI-assisted, community-driven counterspeech, which helps cultivate more effective and motivated counterspeakers, contributing to higher-quality public discourse on pressing societal issues.
- [11] arXiv:2607.28544 [pdf, other]
-
Title: Effects of Auditory Information for People With Visual Impairments in Highly Automated VehiclesComments: Accepted to AutomotiveUI 2026Subjects: Human-Computer Interaction (cs.HC)
Automated vehicles promise to improve accessibility and access to personal mobility for everyone. However, their design and current research trends in visualizing relevant information do not reflect this commitment to accessibility for users with visual impairments. Therefore, we designed and implemented a visual and auditory communication concept for people with visual impairments seated inside fully automated vehicles. Furthermore, in an online video-based study (N=35, 12 with visual impairments), we compared three levels of auditory information communication: low (safety-relevant information only), medium (additionally including vehicle control and route updates), and high (additionally including sightseeing and destination information). Results showed that trust and user experience significantly improved with additional information, with a corresponding, albeit less robust, effect on perceived safety. However, they also revealed that a potential information saturation was reached with medium information. Our work helps to improve the accessibility of automated vehicles by guiding designers towards adequate information communication.
- [12] arXiv:2607.28552 [pdf, other]
-
Title: Multi-Session User Experience Assessments of Computationally Optimized Automated Vehicle Functionality VisualizationsComments: accepted to AutomotiveUI 2026Subjects: Human-Computer Interaction (cs.HC)
Understanding automated vehicles (AVs) is crucial to improving their acceptance. Numerous approaches to visualizing relevant traffic information to passengers have been proposed and empirically evaluated. As this is time-consuming, costly, and reduces the possible design parameters, we employed multi-objective Bayesian optimization to optimize the design of visualizations in AVs. In particular, we evaluated multi-session aspects involving iterative optimization. We optimized the design for passenger trust and perceived safety while minimizing cognitive load. Results from an online study (N=74) show that this method effectively identifies visualization design parameter values that improve trust, safety, and predictability while making the design process more efficient and scalable. However, shortcomings of the computational approach when optimizing for subjective measurements are highlighted and discussed.
- [13] arXiv:2607.28583 [pdf, html, other]
-
Title: CrossAtlas: Evaluating Projection Techniques for Spatial Referencing in Cross-Reality CollaborationHaoyang Yang, Chenyang Zhang, Elliott H. Faa, Weijian Liu, Lily Seika Chisholm, Benjamin Lee, David Saffo, Feiyu Lu, Blair MacIntyre, Yalong YangComments: 11 pages, 8 figures, Accepted to IEEE ISMAR 2026 (TVCG journal track)Subjects: Human-Computer Interaction (cs.HC)
Cross-reality collaboration increasingly connects immersive and desktop users within synchronized workspaces, yet little is known about how bidirectional projection techniques between immersive 3D layouts and desktop 2D views influence communication. Spatial referencing depends on shared spatial understanding, but different mappings preserve and distort geometric relationships in different ways, altering perceived adjacency, orientation, and coverage across collaborators' views. We present CrossAtlas, a synchronized PC-VR collaboration platform that integrates multiple bidirectional projection techniques, including three planar projection variants and equirectangular, a spherical projection variant, across layouts of varying curvature. In a controlled study with 24 dyads, collaborators completed spatial referencing tasks under different projection-layout conditions while we collected performance and subjective measures. Our results show that projection choice strongly shaped collaboration, with the spherical variant often outperforming planar projections and remaining robust across object layouts.
New submissions (showing 13 of 13 entries)
- [14] arXiv:2607.27463 (cross-list from cs.LG) [pdf, other]
-
Title: FADEx: Feature Attribution and Distortion-based Explanation of Dimensionality ReductionComments: 18 pages, 17 figures, to be published in IEEE Transactions on Visualization and Computer GraphicsSubjects: Machine Learning (cs.LG); Human-Computer Interaction (cs.HC)
Dimensionality Reduction (DR) is a fundamental tool for high-dimensional data exploration, reducing the complexity of latent spaces of machine learning models, and assisting in the explanation of complex opaque models. However, non-linear DR techniques often function as opaque transformations themselves, making it challenging to understand how individual features influence instance positioning in the reduced space. This lack of transparency complicates the analysis and interpretation of structural patterns, hindering the ability to reason about the organization of high-dimensional data based on the projected layout. In order to address this challenge, dimensionality reduction explanation methods have shown promise in improving the understanding of the observed groups and cluster structures. Unfortunately, existing DR explanation approaches tend to suffer from limitations such as multiple attributions per feature and restricted applicability to specific dimensionality reduction methods, which hinder their use. In this work, we propose FADEx, a novel local per-instance feature attribution method that leverages local linear approximation via first-order Taylor expansion and Singular Value Decomposition to provide explanations. FADEx computes the local linear models via weighted least squares, eliminating the need for out-of-sample data mapping, making it agnostic to the DR method, while simultaneously providing local feature attributions and distortion analysis. Through qualitative and quantitative evaluations, comparisons with existing methods, and case studies, we demonstrate FADEx's effectiveness and versatility in providing explanations and analytical resources for analyzing the behavior of DR methods. The results indicate FADEx yields robust and reliable explanations, outperforming existing approaches in several aspects.
- [15] arXiv:2607.27565 (cross-list from cs.LG) [pdf, html, other]
-
Title: A Montage-Agnostic Encoder for Calibration-Light Cross-User Gesture Recognition from Surface ElectromyographyComments: 22 pages, 5 figures, 7 tablesSubjects: Machine Learning (cs.LG); Human-Computer Interaction (cs.HC)
Pattern-recognition control promises a myoelectric prosthesis that responds to many intended gestures rather than one or two, but the promise has stayed in the laboratory. A recogniser trained on one person rarely transfers to the next, and useful performance usually demands a fresh round of labelled calibration from the end user. A montage-agnostic encoder is introduced that reads each electrode with shared weights and locates it by its physical coordinate rather than its index, so one architecture ingests any channel count without montage-specific parameters. Trained across users, it exceeds a per-user Hudgins and linear-discriminant classifier by 0.234 macro-F1 on DB1 for every held-out subject and by 0.108 on DB2, and falls below it on the ten-subject DB5. Each of the encoder's three key components individually accounts for more than half of its 3-shot macro F1 in an otherwise budget-matched ablation study. A controlled subject-count sweep shows the margin is close to flat from nine training subjects to thirty-nine, so the training pool binds only as a stability floor below which cross-user training fails to converge; what tracks the direction of the comparison across the three databases is instead the strength of the per-user baseline, which signal fidelity sets. Comparing against an LDA baseline depends on budget spent training models and on how good that baseline is, and self-supervised pretraining had no benefits once a supervised model was adequately trained.
- [16] arXiv:2607.27568 (cross-list from cs.LG) [pdf, html, other]
-
Title: Recognition and Label-Free Adaptation Across Recording Sessions in Surface-EMG Gesture DecodingComments: 23 pages, 6 figures, 6 tablesSubjects: Machine Learning (cs.LG); Human-Computer Interaction (cs.HC)
Recognition accuracy obtained during a recording session does not persist when a user puts on the electrodes again after the electrodes had previously been removed. The electrodes may have moved slightly, the skin may be drier or wetter, or the elbow may be positioned differently; these factors all contribute to day-to-day variability and therefore represent a major obstacle to implementing successful pattern-recognition based myoelectric control systems in daily practice. However, simply recalibrating a user's hand for 20 min at every doff/don event is a clearly unrealistic expectation. A montage-agnostic encoder built for cross-user, cross-montage transfer is trained here using data collected during a particular recording session, and then applied to data collected later in a different recording session without adjusting anything, on the ten intact subjects of NinaPro DB6. The performance of this approach is compared to that of a per-user LDA classification pipeline, and to that of two published approaches that only rely on source data collected from the same recording session. Carried unchanged across recording sessions, the encoder retains 0.688 macro-F1 against 0.540 for the per-user pipeline, and, on the per-window metric the published baselines use, sits above both published source-only results, a band of two points that locates the encoder rather than ranking it. Of five label-free test-time adaptations, only feature-statistic alignment improves every subject; batch-normalisation re-estimation, a standard method in the domain-adaptation literature, collapses this architecture entirely. Aligning the encoder's feature statistics to the new session recovers about what a single labelled calibration repetition would.
- [17] arXiv:2607.27739 (cross-list from cs.IR) [pdf, html, other]
-
Title: Measuring Alignment With Reader Highlights Net of Position and LengthComments: 15 pages, 7 tables. Analysis code and de-identified artifacts included as ancillary files; five of six scripts reproduce the paper's numbers from the shipped artifacts alone. Reports claims from our own prior work that this corpus does not reproduce, and lists twelve claims withdrawn during internal adversarial review in Appendix ASubjects: Information Retrieval (cs.IR); Computation and Language (cs.CL); Human-Computer Interaction (cs.HC)
Context compression discards most of a document before a language model reads it, and is normally evaluated by downstream task accuracy - which makes another model the judge of what mattered. Naturalistic social highlighting offers a non-circular reference: many people independently marking passages on the same page. But the obvious metric, the fraction of crowd-marked sentences a compressor keeps, is confounded twice: crowd marks are front-loaded and crowd-marked sentences are longer, so any method favouring early or long sentences scores well regardless of readers. We remove both by matching each marked sentence against unmarked sentences of the same document at equal relative depth and equal within-document length rank, and we calibrate every estimator on synthetic nulls built from position and length alone - a step that matters, since depth-only stratification returns a false positive on 20-36% of nulls containing no effect. On 120 web documents (at least 12 independent readers each), a language-model importance ranking keeps 38.4% of crowd-marked sentences against 19.9% of their matched neighbours: an enrichment of +0.196 [+0.148, +0.239], at p = 0.0005 under an exact randomization test that assumes nothing about clustering, and replicated cross-vendor. Naive truncation, whose keep rule is position, correctly falls to +0.003. To give the number a scale: scored identically, on the same budget, against a crowd label recomputed to exclude them, a single human reader reaches +0.182 - indistinguishable from GPT-5.4 (+0.002 [-0.081, +0.088]) and below Claude Opus 5. Classical methods are not null - Luhn's 1958 heuristic reaches +0.088 - so reader selection is partly recoverable by counting words; conditioning additionally on lexical centrality removes only 0.010, so the agreement is not centrality. We also report that a claim in our own prior work does not reproduce on this corpus.
- [18] arXiv:2607.27851 (cross-list from cs.CL) [pdf, html, other]
-
Title: Beyond Feeling Better: Capability-Sustaining Emotional Dialogue as a Longitudinal Research ParadigmSubjects: Computation and Language (cs.CL); Human-Computer Interaction (cs.HC); Social and Information Networks (cs.SI)
Emotional dialogue research includes two influential strategy traditions. Empathetic dialogue prioritizes understanding a speaker's emotional experience. Emotional support conversation selects and sequences support for the seeker's current needs. Sustained use introduces a further goal. Effective support should sustain users' capacities for emotion regulation, coping, self-endorsed decisions, and social connection across the interaction lifecycle. We propose capability-sustaining emotional dialogue (CSED) as a longitudinal research paradigm that aligns supportive strategy with this goal and organizes data, models, system design, evaluation, and governance around repeated use, non-use, transition, and termination. A targeted literature-and-corpus audit motivates this position. In a PRISMA-ScR-guided sample, 95% of 60 system-building papers pursue relief-oriented goals. None evaluates capability or longitudinal outcomes, and only 1 considers dependency, autonomy, or termination risk. In 300 ESConv supporter turns, capability-relevant functions appear in 43.0%, while generic suggestions account for 22.0%, compared with 4.0% reappraisal, 6.7% self-efficacy support, and 0.3% boundary behavior. We release a protocol for extending the audit to model behavior. An illustrative process model connects latent user capability to six design commitments, four evaluation timescales, and lifecycle constraints. The resulting agenda makes CSED testable across data, policy design, training, evaluation, and governance.
- [19] arXiv:2607.28302 (cross-list from stat.AP) [pdf, html, other]
-
Title: Snapshot plots: displaying summary tables as parallel univariate plots with consistent color highlightingSubjects: Applications (stat.AP); Graphics (cs.GR); Human-Computer Interaction (cs.HC)
For empirical studies, social and health scientists give background characteristics of their sample and summarize them in the famous "Table 1". When treatment/ control groups are present, this table gives summary statistics by group to see whether the background characteristics differ by group. We propose snapshot plots --- parallel univariate plots with consistent highlighting --- to visualize such tables. Compared to "Table 1", such plots are designed to facilitate comparisons of background characteristics --- in particular among groups --- and give more detail on numerical variables. We provide a web app as well as a python implementation of snapshot plots. Snapshot plots arise as edge cases of hammock plots (parallel coordinate plots for mixed categorical/ numerical data). We demonstrate the usefulness of snapshot plots for two "Table 1"s.
- [20] arXiv:2607.28617 (cross-list from cs.AI) [pdf, html, other]
-
Title: AISPA: User-Centric System Prompt Auditing for Large Language Model ApplicationsXiangning Lin, Shenzhe Zhu, Shu Yang, Zhenyu Zhang, Haoqian Zhang, Yipeng Zhao, Chengxuan Qian, Tianwei Wang, Ziheng Zhang, Zhenlong Yuan, Dingcheng Wang, Juncheng Wu, Yuan Si, Jiaxin Liu, Baolong Bi, Robert Mahari, Tobin South, Dazza Greenwood, Zexue He, Rishi Bommasani, Sophia Kazinnik, Andreas Haupt, Samuele Marro, Erik Brynjolfsson, Alex Pentland, Jiaxin PeiSubjects: Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Computers and Society (cs.CY); Human-Computer Interaction (cs.HC)
System prompts are instructions configured by developers to govern the behaviors of foundation models in AI applications. They are used throughout commercial AI products, but are rarely disclosed to the public or regulators, creating a serious trust and accountability gap in the wide deployment of AI systems. In this paper, we introduce Artificial Intelligence System Prompt Assurance (AISPA), a user-centric framework for systematically auditing system prompts in AI systems. AISPA examines specific parts of a system prompt and evaluates them along eight dimensions that matter to users. We then use this framework to review 3,249 instructions from system prompts in 88 commercial AI products, classifying each instruction as either protective (of users) or problematic. Our audit surfaces four core findings. First, system prompt design varies substantially across products and developers, with some organizations averaging over 60 protective instructions per product while others average fewer than 5. Second, protective instructions are widely adopted but shallow in scope: 98.9% of products contain at least one, yet only 24% cover all eight dimensions of the AISPA taxonomy. Third, system prompts have grown steadily longer and more protective of users, suggesting that user protection is becoming a more visible concern in commercial prompt design. Fourth, despite this progress, problematic instructions remain pervasive: roughly 40% of products contain at least one instruction that works against user interests, and protective and problematic instructions frequently coexist within the same prompt. Our findings highlight the need for greater transparency, standardization, and independent oversight for system prompts in commercial AI products.
Cross submissions (showing 7 of 7 entries)
- [21] arXiv:2402.17838 (replaced) [pdf, other]
-
Title: Personalizing Privacy Protection With Individuals' Regulatory Focus: Would You Preserve or Enhance Your Information Privacy?Reza Ghaiumy Anaraky, Yao Li, Hichang Cho, Danny Yuxing Huang, Kaileigh A. Byrne, Bart Knijnenburg, Oded NovComments: Updated title to match the published CHI 2024 version; substantive content unchangedJournal-ref: Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, Article 982, pp. 1-17 (2024)Subjects: Human-Computer Interaction (cs.HC)
In this study, we explore the effectiveness of persuasive messages endorsing the adoption of a privacy protection technology (IoT Inspector) tailored to individuals' regulatory focus (promotion or prevention). We explore if and how regulatory fit (i.e., tuning the goal-pursuit mechanism to individuals' internal regulatory focus) can increase persuasion and adoption. We conducted a between-subject experiment (N = 236) presenting participants with the IoT Inspector in gain ("Privacy Enhancing Technology" -- PET) or loss ("Privacy Preserving Technology" -- PPT) framing. Results show that the effect of regulatory fit on adoption is mediated by trust and privacy calculus processes: prevention-focused users who read the PPT message trust the tool more. Furthermore, privacy calculus favors using the tool when promotion-focused individuals read the PET message. We discuss the contribution of understanding the cognitive mechanisms behind regulatory fit in privacy decision-making to support privacy protection.
- [22] arXiv:2501.11840 (replaced) [pdf, other]
-
Title: AI-Assisted Data Extraction for Systematic Reviews in EducationSubjects: Human-Computer Interaction (cs.HC)
Systematic reviews are time-consuming endeavors that require knowledgeable human reviewers to screen studies for relevance and extract data following a specific coding scheme before any analysis or synthesis can occur. Large language models (LLMs) hold promise for substantially accelerating this process and reducing reviewer workload, yet their application within the context of systematic reviews in the field of education remains underexplored. We address this issue in two ways: through empirical studies and the iterative development of an open-source software tool. First, we conducted two empirical studies examining the efficacy of using LLMs for data extraction using data from a published review on pedagogical agents. We extracted a variety of data types from 112 studies and compared the results to data extracted by human coding. Results indicate that LLMs struggled with extracting data accurately and therefore are not ready to be used as primary data extraction tools without explicit human validation of the data extracted. These findings highlight the dire need for a human-in-the-loop (HIL) approach to AI-assisted data extraction. We then propose a HIL workflow and introduce and describe the development of a free, web-based, open-source tool designed to support user-friendly, human-validated data extraction with LLMs.
- [23] arXiv:2510.26172 (replaced) [pdf, html, other]
-
Title: Linking Heterogeneous Data with Coordinated Agent Flows for Social Media AnalysisSubjects: Human-Computer Interaction (cs.HC); Artificial Intelligence (cs.AI); Social and Information Networks (cs.SI)
Social media platforms generate volumes of heterogeneous data, capturing user behaviors, textual content, and network structures. Analyzing such data is crucial for understanding phenomena such as opinion dynamics, community formation, and information diffusion. However, discovering insights from this complex landscape is exploratory, conceptually challenging, and requires expertise in social media mining and visualization. Existing automated approaches, including large language models (LLMs), remain largely confined to structured tabular data and cannot adequately address the heterogeneity of social media analysis. We present SIA (Social Insight Agents), an LLM agent system that links heterogeneous multi-modal data, including raw inputs (e.g., text, network, and behavioral data), mined analytical results, and rendered visual artifacts, through coordinated agent flows. Guided by an insight-oriented taxonomy connecting insight types with suitable mining methods and visualization strategies, SIA adopts a stage-synchronized strategy that proceeds through goal decomposition, query, mining, visualization, and reporting stages. At each stage, it collects prior information to jointly plan and execute agent actions, while the coordinator maintains cross-stage action dependencies and assembles and distributes data to agents. Through quantitative evaluation and case studies supported by an interactive interface, we show that SIA can discover diverse and meaningful insights from social media with opportunities for subsequent reliability assessment.
- [24] arXiv:2604.23749 (replaced) [pdf, other]
-
Title: StateScribe: Towards Accessible Change Awareness Across Real-World RevisitsRuei-Che Chang, Xirui Jiang, Rosiana Natalie, Hao Chen, Vlad Roznyatovskiy, Jianzhong Zhang, Kang G. Shin, Ke Sun, Anhong GuoSubjects: Human-Computer Interaction (cs.HC)
Real-world environments evolve continuously, yet blind and low-vision (BLV) individuals often have limited access to understanding how they change over time. Unexpected or relocated objects, layout modifications, and content updates (e.g., price changes) can introduce safety risks and cognitive burden. While existing visual assistive technologies can describe immediate surroundings, they operate as one-off interactions and lack mechanisms to surface meaningful changes across revisits. Informed by a survey of 33 BLV individuals, we develop StateScribe, a system that supports accessible awareness of real-world changes across revisits. StateScribe employs a dual-layer memory architecture that integrates episodic scene memory and object-centric temporal memory to enable scalable and structured change tracking. It provides both live descriptions of the current scene, and descriptions of what has changed, when and where it occurred across revisits, such as "The shop on your right has a "CLOSED" sign; it was open at this time last week.'' Our evaluation shows that StateScribe maintains high accuracy (F1-score=83.1%) across 11 revisits, while remaining low-latency (mean<1.54s) and memory-efficient (<54MB) across 110 revisits. A user study with nine BLV participants demonstrates that StateScribe improves change awareness across revisits in three real-world locations. Finally, we discuss implications for long-term AI-assisted companions that support broader change observation using multimodal sensing, extend beyond changes to other memory capabilities, and adapt to individual users, intents, and contexts.
- [25] arXiv:2606.04308 (replaced) [pdf, html, other]
-
Title: Creative Reading: Scaffolding Reading for TransformationSubjects: Human-Computer Interaction (cs.HC); Information Retrieval (cs.IR); Social and Information Networks (cs.SI)
Reading augmentation systems increasingly help readers process text at scale. While these tools address real constraints of time and cognitive load, they often implicitly frame reading as information transmission, or "reading to discard," delegating interpretation and effort to the machine. Yet this delegation changes the outcome of reading. For example, in scholarly reading, deciding what a research text implies and why it matters is central to the work of scholarly production. We propose creative reading as an alternative goal: reading augmentation that supports readers in creating both readings and themselves as readers. By putting literary and narrative theories into conversation with scholarly sensemaking and creativity support, we present a provocation-oriented design space for valuing the process of reading as a way of preserving a plurality of readings and transforming readers over time.
- [26] arXiv:2606.26364 (replaced) [pdf, html, other]
-
Title: Having Dog Ears "for Real": Effects of Active and Passive Haptics on Embodying Non-Human Body Parts in VRComments: Accepted to IEEE Transactions on Visualization and Computer Graphics (TVCG)Subjects: Human-Computer Interaction (cs.HC)
Embodying non-human body parts in VR is a prevalent practice among certain subcultures and is a personally important creative outlet to many individuals. However, the discrepant morphology between real and virtual bodies can decrease Sense of Embodiment (SoE). Haptic feedback can compensate by increasing SoE felt towards non-human body parts, but there is a literature gap in comparing the effects of different haptic modalities, and their combinations, on SoE. Through an online survey sent out to social VR communities (n = 63), we determined that animal ears are a commonly embodied and ecologically valid non-human body part to study. We then ran a 2x2 within-subjects user study (n = 28) with two independent variables: active haptics, delivered through vibrotactile gloves, and passive haptics, delivered through a physical headband, for when participants reach up to touch virtual dog ears appended to their avatar in VR. Our findings show that (1) passive haptics produced the strongest overall embodiment outcomes, (2) combining modalities reduced the benefits of passive haptics, and (3) SoE towards non-human body parts positively correlates with SoE towards the entire avatar.
- [27] arXiv:2607.07105 (replaced) [pdf, html, other]
-
Title: CompoVista: A Composition-Graph-Based Visual Analytics System for Compositional Analysis of Traditional Chinese PaintingsDekun Qian, Ruiqi Yu, Li Ye, Yize Li, Fengling Zheng, Weigui Zheng, Yigang Wang, Jinchang Li, Zhiguang ZhouSubjects: Human-Computer Interaction (cs.HC); Graphics (cs.GR)
Compositional analysis of Traditional Chinese Paintings (TCPs) reveals how spatial arrangement, narrative structure, and cultural-aesthetic meaning are organized within the pictorial field. Traditional compositional analysis relies primarily on qualitative interpretation, supporting close examination of individual paintings but offering limited capacity to identify, compare, and validate compositional patterns across large-scale collections. To identify the key challenges in analyzing composition across large TCP collections, we collaborated with two art historians and conducted a complementary literature review. Drawing on the resulting insights, we introduce CompoGraph, a structured representation for composition-oriented analysis of TCPs. It represents the composition of a painting across four layers: entities, relations, voids, and context. Based on this representation, we develop CompoVista, a canvas-based visual analytics system for composition-oriented exploration of TCPs. CompoVista allows art historians to construct and refine painting cohorts through interactive compositional queries. It also supports inspecting entity distributions and relations at the cohort level, comparing compositional differences across cohorts, and tracing aggregate patterns back to painting-level evidence. Through two case studies, a user study, and expert interviews, we demonstrate that CompoVista can help art historians discover, compare, and validate compositional patterns across collections of TCPs.
- [28] arXiv:2607.20835 (replaced) [pdf, html, other]
-
Title: Sonic Stage: Auto-Generating Interactive Spatial Soundscapes to Facilitate Dialogue Video Comprehension for Blind ViewersComments: accepted to UIST 2026Subjects: Human-Computer Interaction (cs.HC)
Audio description (AD) makes film and television accessible to blind and low-vision (BLV) audiences by narrating characters' actions. However, in scenes with lots of dialogue, AD often omits important actions because it is constrained not to overlap with speech. It is not yet known how to convey characters' actions during dialogue. We present Sonic Stage, a system that transforms dialogue videos into interactive spatial soundscapes, enabling BLV audiences to intuitively understand characters' actions and movements through immersive auditory cues. Sonic Stage conveys essential visual information during dialogue through three auditory techniques: (1) spatialized dialogue to represent spatial layout, (2) diegetic sound to convey character actions, and (3) interactive descriptions to provide context-specific visual details. Evaluation with 12 BLV viewers showed that Sonic Stage significantly improved video comprehension, spatial presence, and narrative engagement. We highlight opportunities for enhancing video accessibility across diverse genres through immersive, interactive audio representations.
- [29] arXiv:2607.26659 (replaced) [pdf, html, other]
-
Title: Galvanic Vestibular Stimulation in Latent SpaceSubjects: Human-Computer Interaction (cs.HC); Computer Science and Game Theory (cs.GT)
Galvanic vestibular stimulation (GVS) is widely used to modulate self-orientation, balance, and motion perception; the discriminability of frequency-encoded cues further suggests its potential as a standalone modality for embodied feedback. However, synthesizing GVS waveforms congruent with target events or bodily states remains challenging. GVS waveforms combine current direction, intensity, duration, and onset and offset transitions, yet how these parameters jointly shape users' perceptual and associative responses remains underexplored. To address this gap, we contribute a dataset linking GVS waveforms to free-form experience descriptions, as well as a retrieval-guided generative model for synthesizing candidate waveforms from target descriptions. The dataset comprises 100 GVS waveforms and 1,526 valid free-form sensation descriptions collected from 16 participants. Semantic analysis revealed diverse motion- and force-related sensations, localized bodily sensations, and situational associations. Compared with a participant-preserving permutation baseline, descriptions elicited by the same waveform covered fewer semantic categories (8.18 vs. 9.45) and exhibited a higher dominant-category proportion (26.97% vs. 21.25%; both P < 0.001). Building on this dataset, we implemented the generative model as a retrieval-guided one-dimensional convolutional variational autoencoder. An independent behavioral study recruited 10 participants who had not contributed to the dataset collection. Performance in discriminating congruent from incongruent waveform-visual cue pairings was significantly above chance, with an accuracy of 63.33%, d-prime = 0.70, and p < 0.001. Together, these findings demonstrate the feasibility of text-conditioned GVS synthesis and support the development of GVS as a programmable modality for semantically congruent embodied feedback across interactive scenarios.
- [30] arXiv:2509.12290 (replaced) [pdf, html, other]
-
Title: Secure human oversight of AI: Threat modeling in a socio-technical contextJonas C. Ditz, Veronika Lazar, Elmar Lichtmeß, Carola Plesch, Matthias Heck, Kevin Baum, Markus LangerSubjects: Cryptography and Security (cs.CR); Computers and Society (cs.CY); Human-Computer Interaction (cs.HC)
Human oversight of AI is promoted as a safeguard against risks such as inaccurate outputs, system malfunctions, or violations of fundamental rights, and is mandated in regulation like the European AI Act. Yet debates on human oversight have largely focused on its effectiveness, while overlooking a critical dimension: the security of human oversight. We argue that human oversight creates a new attack surface within the safety, security, and accountability architecture of AI operations. Drawing on cybersecurity perspectives, we model human oversight as an IT application for the purpose of systematic threat modeling of the human oversight process. Threat modeling allows us to identify security risks within human oversight and points towards possible mitigation strategies. Our contributions are: (1) introducing a security perspective on human oversight, (2) offering researchers and practitioners guidance on how to approach their human oversight applications from a security point of view, and (3) providing a systematic overview of attack vectors and hardening strategies to enable secure human oversight of AI.
- [31] arXiv:2512.06354 (replaced) [pdf, html, other]
-
Title: The Missing Variable: Socio-Technical Alignment in Risk EvaluationComments: This paper was accepted for the IFAC World Congress 2026Subjects: Computers and Society (cs.CY); Human-Computer Interaction (cs.HC); Systems and Control (eess.SY)
This paper addresses a critical gap in the risk assessment of AI-enabled safety-critical systems. While these systems, where AI systems assist human operators, function as complex socio-technical systems, existing risk evaluation methods fail to account for the associated complex interaction between human, technical, and organizational components. Through a comparative analysis of system attributes from both socio-technical and AI-enabled systems and a review of current risk evaluation methods, we confirm the absence of explicit socio-technical considerations in standard risk expressions. To bridge this gap, we introduce a novel socio-technical alignment ($STA$) variable designed to be integrated into the traditional risk equation. This variable estimates the degree of harmonious interaction between the AI systems, human operators, and organizational processes. A case study on an AI-enabled liquid hydrogen ($LH_2$) bunkering system demonstrates the variable's relevance. By comparing a naive and a safeguarded system design, we illustrate how the $STA$-augmented expression captures socio-technical safety implications that traditional risk evaluation overlooks, providing a more system-theoretic basis for risk evaluation.
- [32] arXiv:2603.26487 (replaced) [pdf, html, other]
-
Title: To Ban or not to Ban? How Open Source Projects Govern GenAI ContributionsComments: Accepted at the 41st IEEE/ACM International Conference on Automated Software Engineering (ASE 2026)Subjects: Software Engineering (cs.SE); Human-Computer Interaction (cs.HC)
Generative AI (GenAI) is playing an increasingly important role in open source software (OSS). Beyond completing code and documentation, GenAI is increasingly involved in issues, pull requests, code reviews, and security reports. Yet, cheaper generation does not mean cheaper review - and the resulting maintenance burden has pushed OSS projects to experiment with GenAI-specific rules in contribution guidelines, security policies, and repository instructions, even including a total ban on AI-assisted contributions. However, governing GenAI in OSS is far more than a ban-or-not question. The responses remain scattered, with neither a shared governance framework in practice nor a systematic understanding in research. Therefore, in this paper, we conduct a multi-stage analysis on various qualitative materials related to GenAI governance retrieved from 67 highly visible OSS projects. Our analysis identifies recurring concerns across contribution workflows, derives three governance orientations, and maps out 12 governance strategies and their policy instruments. We show that governing GenAI in OSS extends well beyond banning - it requires coordinated responses across accountability, verification, review capacity, code provenance, and platform infrastructure. Overall, our work distills dispersed community practices into a structured overview, providing a conceptual baseline for researchers and a practical reference for maintainers and platform designers.
- [33] arXiv:2607.09306 (replaced) [pdf, html, other]
-
Title: Exposure is not manifestation: measurement target and output resolution jointly determine which behavioural-faithfulness evaluator winsComments: Substantially revised and narrowed version with a new title and estimand-centred analysis. Comparisons are now reported at three output resolutions, and the reproducibility package has been rebuilt. The author list was changed with the approval of all authors listed on v1-v2; previous versions remain publicly available. 17 pages, 3 figures, 3 tablesSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC); Machine Learning (cs.LG)
Behavioural auditing asks whether a language model behaves as it claims, but detection scores are reported without separating two targets: whether a reply was produced under a behaviour-inducing condition (exposure) and whether the behaviour surfaced in it (manifestation). Scoring a compact 146-million-parameter auditor's frozen-representation read-out and a frontier judge against each label on the identical 720 replies, the gap between the instruments moves by roughly 0.2 AUROC when the target changes. Under the judge's deployed interface, a single verdict, the ranking reverses: the auditor leads on exposure, 0.804 against 0.718, and trails on manifestation, 0.690 against 0.811. Matching the output resolution from either direction, by asking the judge a target-specific question answered with a continuous confidence score or by thresholding the auditor's read-out, removes the reversal but not the interaction, which excludes zero at all three resolutions (0.207, 0.237 and 0.169). The target governs how far apart the instruments are; the interface governs whether that distance changes their order. The auditor's hyperbolic geometry confers no advantage here. A single behavioural-detection AUROC is under-specified: such claims are comparable only when they state the estimand, the evaluator, and its output interface.
- [34] arXiv:2607.13220 (replaced) [pdf, html, other]
-
Title: Networked Intelligence: Active Shared Context Graphs for Human-AI Team ScienceSutanay Choudhury, Jeffrey J. Czajka, Lummy M. O. Monteiro, Erin Bredeweg, Jason McDermott, Katherine Wolf, Alex Beliaev, Josh Elmore, Paul Piehowski, Kylee Tate, Yuqian Gao, Aivett Bilbao, Kelly Stratton, Scott Baker, Jaydeep P. Bardhan, Kristin Burnum Johnson, Chris Oehmen, Robert RalloSubjects: Artificial Intelligence (cs.AI); Computational Engineering, Finance, and Science (cs.CE); Human-Computer Interaction (cs.HC)
Most AI-for-science systems focus on scaling a single reasoning process by using better models, larger context windows, long-horizon agentic execution, or digital co-scientists working with one principal user. However, challenging scientific problems are rarely solved by one reasoner alone. They are solved by teams whose members carry different priors, experimental background, tacit knowledge, and domain-trained intuitions. The open problem is therefore not only how to scale models, but how to develop "networked intelligence", scaling the connections between humans and AI systems so that a result or hypothesis produced in one context reaches another person, agent, instrument or robot that can act on it. We introduce Mycelium, an active shared workspace that automatically connects researchers and AI agents. As human users and agents work, the system captures important observations and hypotheses, tracks how they relate to the team's evolving knowledge model, and routes them to the person or agent whose next decision they can inform. We evaluate Mycelium through a real-world scientific discovery use case: a biological multi-omics campaign where shared context turned a local analytical finding into a cross-expert mechanistic constraint and ultimately into an experimental design. Finally, we describe networked intelligence as sparse conditional computation over distributed scientific contexts. This framework establishes when a scaled standalone agent is sufficient, and when isolated data and specialized expertise make a networked approach essential.
- [35] arXiv:2607.27189 (replaced) [pdf, html, other]
-
Title: APEX-AccountingJulien Benchek, Austin Bennett, Jasmin Kern, Ryan Stevens, Rene Sultan, Charis Ching, Hayley Popiel, Vaibhav Mittal, Felix Mercier, Brendan Foody, Bertie VidgenComments: Public dev set: this https URLSubjects: Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Human-Computer Interaction (cs.HC)
We introduce APEX-Accounting, a benchmark built by Mercor in partnership with Ramp, to assess whether frontier models can do the real work of accountants. Tasks include reconciling accounts, accruing expenses, posting transactions, and producing reports. The private eval set comprises 160 tasks, split across 10 worlds. Each world contains an accounting system, as well as spreadsheets, PDFs, and other files. Every task was authored and solved by experts in accounting and bookkeeping, who also wrote grading rubrics. Across nine frontier models, Claude-Fable-5 (Max) leads with 56.4% Mean Criteria@3, ahead of Muse-Spark-1.1 (xHigh) at 52.6%. No model scores more than 2.6% Pass^8 (GPT-5.6-Sol (Max+Pro)) and the highest Pass@8 is 21.5% (Muse-Spark-1.1 (xHigh)). We experiment with increasing the token budget from $1 to $50 and observe an instance of Simpson's paradox: scores increase as the token budget increases but within a given budget-constrained harness, scores are lower on tasks where the model spends more tokens. As APEX-Accounting is a closed benchmark, leaderboard evals can be run for any frontier model on request.