"Hi Alex" or "Dear Dr. Morgan"?: Exploring How AI-suggested Politeness Strategies Influence Email Writing and Social Perception Among Native and Non-native Speakers

3.2.1 Writing Task and Tool. Each participant completed three writing tasks, where they wrote emails based on three scenarios involving professional requests. They were asked to imagine themselves in the scenario as the person making the request, or to recall a previous experience in a similar situation.

In all three writing tasks, participants used three different variations of a writing tool that shared a similar interface, as shown in Figure 1. In the baseline condition, participants were provided with only a text box without AI suggestion features. They were discouraged from using any external AI tools, and were informed that if the use of such tools was detected, they may be denied remuneration. In the positive-AI and negative-AI conditions, participants were provided with two AI writing assistants that either favour positive politeness strategies or negative politeness strategies in the suggestions. The tools provided text continuation suggestions (approximately 20–30 words) based on existing content written by the user. The AI writing assistant was also prompted with the respective social scenario to ensure that the language suggested by the AI writing assistant was appropriately adapted to the scenario. We prompted the LLM with the scenarios to ensure that participants would receive useful suggestions for their tasks. In practice, people often get writing assistance from LLMs that are adapted to their task—either using domain-specific writing assistance for which the developers performed the adaptation, or providing the context in their own prompts. To avoid the individual variance in their prompting skills, we added the scenarios in the system prompt to create task-adapted writing assistance.

The interface modelled AI assistant interactions in popular email applications and previous studies [27]. Specifically, text continuation suggestions were automatically generated after participants pressed the [space] or [enter] and paused typing for 1.5 seconds. This design ensured that suggestions do not disturb participants’ writing flow when they are continuously typing. If participants were not satisfied with the suggestions, they could continue writing to ignore suggestions or press the [tab] key to re-generate suggestions. Participants were encouraged to use as many AI suggestions as they deemed appropriate.

Participants were motivated to produce high-quality emails through both social and monetary incentives. Before completing the baseline and AI writing tasks, they were provided with the following instruction: “Your email will be reviewed by individuals who could realistically be the recipients. These reviewers will rate the email's social appropriateness and the likelihood of providing a favorable response. The top 10% of participants with the highest ratings across three emails will receive an additional £3 reward.” The inclusion of bonus rewards follows common practice in HCI to ensure quality control in studies that use online crowdsourcing platforms [19] [40] [10]. The performance-based bonus served as a monetary incentive, while emphasizing evaluation by plausible recipients functioned as a social incentive.

Fig. 1. The interface participants used in the writing tasks. 1: Participants start by writing the initial opening of the email; 2: AI suggestions can be called to continue the human-written components; 3: the “...Generating...” sign indicates that the system is working on generating suggestions.

3.2.2 AI Model Configuration. The AI writing assistants in the positive-AI and negative-AI conditions differ in the politeness strategies included in the suggestions. These two AI assistants were configured differently to provide suggestions that either predominantly include positive politeness strategies or negative politeness strategies.

The suggestions were generated by calling the gpt-4o-2024-08-06 API. The difference between conditions was achieved through prompting. The prompts included the definition and examples of the politeness strategies based on [11] and [41]. The full list of politeness strategies with descriptions and examples is included in Appendix A.

To validate the politeness manipulation, we conducted two manipulation checks. In the first step, we examined whether emails generated using positive politeness prompts, negative politeness prompts, and the default GPT-4o model (without additional prompt modifications) were distinguishable on key manipulated variables at an overall level. The emails used in this test were generated in varied scenarios, including low-stakes interactions between friends, high-stakes email requests related to fundraising, and middle scenarios between the two extremes. The manipulation check was done through human rating on 12 AI-generated emails: 4 using the default GPT-4o model without additional prompts, 4 with positive politeness prompts, and 4 with negative politeness prompts. Two raters were provided with definitions and descriptions of positive and negative politeness strategies and were instructed to categorize the emails into the three categories: default, positive, and negative. Rater 1 correctly categorized the emails with (75%) accuracy, while Rater 2 achieved (67%) accuracy. The most common error was misclassifying default emails as negative politeness emails. This is consistent with previous work [15] suggesting that GPT's default language style aligns more closely with negative politeness strategies. Both raters achieved (87.5%) accuracy in distinguishing between positive-AI and negative-AI, indicating that the manipulation was effective.

To further validate the strength of the manipulation, we conducted a second manipulation check that compared the number of positive and negative strategies present in the generated emails. To closely mirror the study stimuli, we generated 18 emails based on the writing scenarios used in the experiment (see Section 3.2.3). The dataset comprised three scenarios, three conditions (positive politeness, negative politeness, and default GPT-4o), and two emails per scenario–condition combination. Two independent coders applied the codebook to identify all positive and negative strategies in each email. We assessed interrater agreement using the Intraclass Correlation Coefficient (ICC), focusing on ICC(3,k) because the final scores reflect the average of two fixed raters. The resulting ICC(3,k) was 0.88 for negative strategies and 0.77 for positive strategies, both exceeding the threshold for good reliability according to Koo and Li's guidelines [31].

ANOVA results showed a significant effect of AI politeness condition (positive, negative, default) on strategy use for both negative strategies, F(2, 9) = 163.20, p <.001, and positive strategies, F(2, 9) = 71.73, p <.001. Post hoc t-tests further clarified these effects. Compared with the default condition, emails generated with negative-strategy prompts contained significantly more negative strategies [M_neg = 21.75, SD = 4.87 vs. M_default = 9.50, SD = 2.37; t = 5.55, p <.001], but showed no significant increase in positive strategies [M_pos = 4.83, SD = 1.51 vs. M_default = 4.00, SD = 1.61; t = 0.93, p =.377]. A parallel pattern was observed for positive-strategy prompts: compared with the default condition, they produced significantly more positive strategies [M_pos = 11.75, SD = 4.19 vs. M_default = 4.00, SD = 1.61; t = 4.23, p =.002], but no significant difference in negative strategies [M_neg = 10.67, SD = 2.21 vs. M_default = 9.50, SD = 2.37; t = 0.88, p =.398]. Together, these findings confirm successful manipulation.

3.2.3 Scenarios. The scenarios used in the study include professional situations that involve making a request. Two sets of scenarios were used in the study — three scenarios were used in writing tasks to guide email writing, and they were slightly modified and mixed with seven additional scenarios to collect participants’ social perception before writing tasks. The purpose of using two sets of scenarios was to reduce anchoring effects.

We initially created 18 scenarios adapted from previous work [20], and they broadly fall into three categories: high-risk situations (high D, P, R values), low-risk situations (low D, P, R values), and medium-risk scenarios that are more ambiguous. To validate that the scenarios are perceived as we intended, we performed a pilot study with 18 participants (N = 18) who rated all scenarios along the three perception dimensions (D, P, R). The results confirmed that perceptions towards the scenarios generally aligned with our intentions, and we selected three scenarios from the medium-risk category with the most variance across participants to use as writing tasks. We chose these scenarios because they are more open to interpretation, thus there is potentially more room for AI's influence. The three writing scenarios (WS1–3) are listed as follows:

• WS1: You're enrolled in a course and have interacted with the course TA (teaching assistant) during office hours. You write an email to ask if the TA could clarify a specific concept from a recent class discussion or recommend some additional study materials related to the topic.
  
• WS2: You are working on a project with a colleague, and you're at a stage where you need their part of the work to be completed in order to proceed with your own tasks. You write an email about their progress and to find out if they can finish their part within the week to keep the project on schedule.
  
• WS3: You're working on a group project for a university course and have had a few discussions with your project advisor. You write an email to ask if the advisor could provide feedback on your current project outline or suggest any resources that might help improve your presentation.
  

We additionally selected seven scenarios to form the second set of scenarios used to collect pre-writing social perception. Among them, two were selected from the high-risk category, two from the low-risk category, and three from the medium-risk category. The writing scenarios were also slightly modified in the pre-writing survey scenarios. The full list of pre-writing survey scenarios and descriptions is included in Appendix B, and scenarios that correspond to writing scenarios are highlighted.

3.5.1 Change in Politeness Strategies. To study H1, we measured changes in politeness strategies as the dependent variable. For each participant, four different scores were calculated by comparing the number of positive and negative strategies used in the positive-AI and negative-AI conditions to those in the baseline condition. The four scores are defined as follows using mathematical notation:

• Change in positive strategies (positive-AI condition):
  \[ \Delta \text{Positive}_{\text{positive-AI}} = \text{\# Positive}_{\text{positive-AI}} - \text{\# Positive}_{\text{baseline}} \]
  
  
• Change in negative strategies (positive-AI condition):
  \[ \Delta \text{Negative}_{\text{positive-AI}} = \text{\# Negative}_{\text{positive-AI}} - \text{\# Negative}_{\text{baseline}} \]
  
  
• Change in positive strategies (negative-AI condition):
  \[ \Delta \text{Positive}_{\text{negative-AI}} = \text{\# Positive}_{\text{negative-AI}} - \text{\# Positive}_{\text{baseline}} \]
  
  
• Change in negative strategies (negative-AI condition):
  \[ \Delta \text{Negative}_{\text{negative-AI}} = \text{\# Negative}_{\text{negative-AI}} - \text{\# Negative}_{\text{baseline}} \]
  
  

The number of positive and negative strategies was determined through qualitative coding of participants’ emails. Each email was manually annotated for instances of positive and negative politeness strategies. The codebook was developed based on Brown & Levinson's politeness theory [11] and was adapted to ensure both comprehensiveness and mutual exclusivity of the codes. The final codebook includes 14 positive politeness strategies and 14 negative politeness strategies.

Using the codebook, two raters, including the first author of the paper, independently coded 20% of emails, which follows conventions in the HCI community [48][26]. To assess interrater reliability, we calculated the Intraclass Correlation Coefficient (ICC), focusing on ICC3 (single fixed raters). For number of negative strategies, the ICC3 was 0.71, indicating moderate reliability. For number of positive strategies, the ICC3 was 0.86, indicating good reliability. The guidelines for interpreting the data were adopted from Koo and Li [31]. These results demonstrate that the interrater reliability is sufficient to support the use of these ratings in the analysis. The first author of the paper then continued to rate the rest of the email samples, and the analysis was based on data coded by the first author.

3.5.2 Social Perception Change. To study H2, the change in social perception was measured by comparing social perception ratings in the post-writing survey and the pre-writing survey (for the scenario that matched the one used in the writing task). The calculation of perception change is represented below:

• Change in Distance perception:
  \[ \Delta D = D_{\text{post-writing}} - D_{\text{pre-writing}} \]
  
  
• Change in Power perception:
  \[ \Delta P = P_{\text{post-writing}} - P_{\text{pre-writing}} \]
  
  
• Change in Imposition perception:
  \[ \Delta R = R_{\text{post-writing}} - R_{\text{pre-writing}} \]
  
  

Social perceptions were measured using a 1–7 self-report Likert scale, with one question assessing each of the dimensions: distance, power, and imposition. Participants were provided with definitions and explanations of these constructs to ensure clarity. The imposition scale was relabeled to use more colloquial terms — “demanding” and “undemanding”, which was adapted from [53] to avoid confusion in understanding the questions. On the scales, a score of 1 aligns with low-risk scenarios — it indicates a close relationship, greater power held by the sender, and less demanding requests. A score of 7 aligns with high-risk situations with a more distant relationship, greater power held by the receiver, and more demanding tasks. The full descriptions of the constructs and the scale are included in the Appendix C.

3.5.3 Additional Variables. Our analyses included three additional variables: reliance on AI, individual's baseline social perception, and writing quality of the emails.

Reliance on AI is a mediating variable used to further investigate the effect of AI politeness and language on change in politeness strategies. Reliance on AI is measured by the ratio between the number of accepted AI suggestions and the number of generated AI suggestions. The number of generated AI suggestions includes the number of generation and re-generation requests from the user. While there are other potential ways to reflect reliance (e.g., the proportion of AI-generated text in the final email), this measure was selected because it reflects participants’ explicit decisions to accept or reject AI suggestions during the writing process. By contrast, final-text proportion reflects how much AI wording is preserved in the final product, which may not directly indicate reliance, as participants may reject a suggestion but later adopt its idea in their own words, or accept a suggestion but revise it heavily before submission.

Baseline social perception is another covariate used to study how different baseline social perceptions might affect social perception change. Baseline social perception is calculated by taking the average of participants’ pre-writing perception ratings across the 10 scenarios on all three dimensions, D, P and R. Variation in this measure reflects individual differences in how people initially interpret and evaluate different social situations.

Email quality is an additional outcome variable included in exploratory analysis. Four raters evaluated the quality of all emails without knowing the conditions the emails were written in. Two of the raters are from the research team, and two raters outside of the research team are the potential receivers of the emails (who have been teaching assistants and instructors for university courses). The raters rated the emails’ writing quality on a 1–7 scale based on a rubric that was adapted from a previous work [28] and [58], which focuses on language accuracy, appropriate email structure including opening and closing, and detailed but succinct delivery of relevant information. The full rubric is included in Appendix D.

We calculated the inter-rater reliability by ICC after rating 10% of emails, and ICC3k (average fixed raters) was (0.80), indicating good reliability between the raters. The raters then independently coded the rest of the emails, and independent reliability was calculated again on the full dataset, with an ICC3k value of (0.78), indicating good reliability. The average rating of the raters was used as the quality rating for analysis.

4.1.1 Effects of AI politeness and language on Change in Politeness Strategies. We tested H1 and H3a, which hypothesized that AI politeness (positive-AI vs. negative-AI) and language (NNES vs. NES) would both influence changes in politeness strategies people used in the emails. Results fully confirmed H1 and partially confirmed H3a.

The outcome variable, number of politeness strategies included in the emails, is measured by qualitatively coding the emails and counting the number of positive and negative strategies included within them. Here we focused on the change in the number of politeness strategies, which is calculated by subtracting the number of strategies in the baseline condition from those in positive-AI and negative-AI conditions, respectively. The full analysis details were included in Section 3.5.1. We ran a mixed-effects model to analyze the main effects of AI politeness and language, and their interaction on changes in politeness strategies, with individual participants treated as a random effect. In all analyses, the negative-AI condition and NNES group were used as the reference levels, unless otherwise specified.

AI politeness has a significant main effect on both the change in positive strategies (ΔPositive) (β = 5.50 , SE = 0.86 , t(51) = 6.40 , p < 0.001) and the change in negative strategies (ΔNegative) (β = −5.46 , SE = 1.09 , t(51) = −5.00 , p < 0.001). Figure indicates that the negative-AI condition resulted in a greater increase in negative strategies, and the positive-AI condition led to a larger increase in positive strategies. This aligns with the expectation and confirms H1 that AI politeness changes the corresponding politeness strategies people include in emails.

There is also a significant main effect of language on ΔPositive (β = −2.52 , SE = 1.19 , t(50) = −2.118 , p = 0.04) and a close to significant interaction effect between AI politeness and language (β = 3.35 , SE = 1.67 , t(50) = 2.004 , p = 0.051). In the interaction effect analysis, when using the negative-AI condition as the reference, the main effect of language on ΔPositive was not statistically significant (β = −0.84, SE = 1.45, t(89.71) = −0.58, p = 0.56). This indicates that NNESs and NESs do not change their use of positive strategies in different ways influenced by the negative-AI. Interestingly, when the reference level switched to the positive-AI condition, the effect of language became significant (β = −4.20, SE = 1.45, t(89.71) = −2.89, p < 0.01). This means that NNESs increased their use of positive strategies in the positive-AI condition to a higher extent compared to NESs, indicating that they are more influenced by AI suggestions.

However, for ΔNegative, no evidence suggests a main effect of language (β = −2.49 , SE = 1.58 , t(50) = −1.58 , p = 0.12) nor an interaction effect between AI politeness and language (β = −0.50 , SE = 2.21 , t(50) = −0.23 , p = 0.82). This suggests that NESs and NNESs are not affected differently in terms of change in negative strategies. The interaction between language and condition for both positive and negative strategies is visualized in Figure 3.

Overall, the results indicate that NNESs increase their positive strategies more than NESs. Specifically, NNESs show a larger increase in positive strategies in the positive-AI condition but not in the negative-AI condition, as indicated in Figure 3a . This supports that AI's use of positive strategies has a larger impact on NNESs than NESs. On the other hand, while Figure 3b indicates that NNESs exhibit a larger change in negative strategies across conditions, no strong statistical evidence supports this trend. These findings offer partial support for H3a, suggesting that AI politeness strategies only influence NNESs more in their use of positive strategies but not their use of negative strategies.

Fig. 3. The change in use of positive and negative strategies is different across AI politeness and language groups.

4.1.2 Effect on the Human-Written Content in the Emails. To rule out the possibility that the change in email politeness strategies was solely due to the adoption of AI-suggested text that included such strategies, we performed further analysis on the human-written content in the emails. Results indicate that AI politeness also significantly affects the number of positive strategies in the human-written content, further supporting H1, while there is only a marginal effect on negative strategies. A marginal interaction effect suggests that AI politeness influences NNESs and NESs differently on their adoption of positive politeness strategies, providing partial support for H3a.

The human-written content was isolated by removing AI-generated content from the final email. AI-generated content was identified by the overlapping words and sentences between AI suggestions and the final email. Using the same qualitative codebook, the total numbers of positive and negative strategies were again quantified from the human-written content. As the human-written content only includes partial emails, calculating changes in politeness strategies by comparing to the baseline condition is no longer methodologically appropriate. Thus, we focus on the total number of positive ($\# \text{Positive}$) and negative strategies ($\# \text{Negative}$) as outcome variables, instead of changes in number of strategies (ΔPositive and ΔNegative). We then applied a similar mixed-effects model to $\# \text{Positive}$ and $\# \text{Negative}$, treating AI politeness and language as fixed effects, with scenario and individual participants as random effects. The negative-AI condition and the NNES group were used as the default reference levels, consistent with previous analysis.

The results show a significant main effect of AI politeness on $\# \text{Positive}$ (β = 1.14 , SE = 0.43 , t(49) = 2.69 , p < 0.01) and a marginal main effect of AI politeness on $\# \text{Negative}$ (β = −1.07 , SE = 0.58 , t(50) = −1.82 , p = 0.07). No significant main effect of language was observed on either $\# \text{Positive}$ β = 0.00 , SE = 0.73 , t(49.33) = 0.01 , p = 1.00 or $\# \text{Negative}$ β = 1.06 , SE = 0.99 , t(50.04) = 1.07 , p = 0.29.

There is marginal evidence for an interaction effect of AI politeness and language on $\# \text{Positive}$ (β = 1.58 , SE = 0.83 , t(48) = 1.90 , p = 0.06). Specifically, when using NNES as the reference group, the effect of the AI politeness was significant (β = 1.97 , SE = 0.60 , t(48.03) = 3.26 , p < 0.01), but when NES is treated as the reference group, the effect of AI politeness is not significant (β = 0.39 , SE = 0.57 , t(47.65) = 0.69 , p =.495) For $\# \text{Negative}$, there is no interaction effect between AI politeness and language (β = −0.42 , SE = 1.19 , t(48.29) = −0.36 , p = 0.72). The data is visualized in Figure 4.

Overall, the results provide additional evidence for H1 and H3a, while also suggesting a need for more nuanced interpretation. They confirm that AI politeness can shape the language choices in parts written solely by the human. However, the impact on negative strategies is only marginal, suggesting that the impact of AI politeness might not be extended to the use of negative strategies in human-written contents.

The marginal interaction effect between AI politeness and language on the number of positive strategies also aligns with what we found on the full AI-assisted email. Results indicate that the influence of AI politeness on the use of positive strategies extends to human-written content for NNES participants, but not for NES participants. Although the interaction effect is only marginal, this also adds support for H3a. On the other hand, aligning with the analysis of the AI-assisted full email, the effect of language on negative strategies is not significant. This means that NNES are not more likely to be influenced byAI politenesson negative strategies. The evidence offers partial support for H3a, highlighting the need for additional research to better understand the effect.

Fig. 4. The use of positive and negative strategies is different across language for human-written content

4.1.3 Mediating Effect of Reliance on Change in Politeness Strategies. We additionally attempted to uncover the mediating factors that contribute to the influence of AI suggestions on email writing. One factor we hypothesize to play a major role is reliance. We hypothesize that the effect of AI politeness is not mediated by reliance, while the effect of language is mediated by reliance. The data support this hypothesis.

Reliance is measured by calculating the acceptance ratio between the number of accepted AI suggestions and the total number of AI suggestions called. Details of this measurement are included in Section 3.5.3.

We first studied whether reliance is affected by AI politeness and language. We ran a linear mixed-effects analysis on reliance, with AI politeness and language as fixed effects, and participant ID and scenario as random effects. The negative-AI condition and the NNES group were set as the reference levels. Results indicate no significant main effect of AI politeness (β = −0.02 , SE = 0.04 , t(49.61) = −0.42 , p = 0.68) but a marginally significant effect of language (β = −0.10 , SE = 0.06 , t(49.02) = −1.80 , p = 0.08). No significant interaction effect was found between AI politeness and language (β = −0.11 , SE = 0.08 , t(48.44) = −1.37 , p = 0.18). However, in the positive-AI condition, a significant difference in reliance was found between NNESs and NESs, with NNESs being more reliant on AI suggestions (β = −0.15 , SE = 0.07 , t(86.10) = −2.26 , p = 0.03). There is no significant difference between language groups in the negative-AI condition (β = −0.05 , SE = 0.07 , t(86.92) = −0.71 , p = 0.48). The results are also shown in Figure 5.

Fig. 5. NNESs are more reliant on AI when it predominantly suggests positive strategies.

Although the interaction between AI politeness and language group was not statistically significant, the effect of language was significant in the positive-AI condition but not in the negative-AI condition. This suggests that when AI suggestions predominantly include positive strategies, the NNES group is more reliant on AI than the NES group. This is congruent with our other findings, which suggest the effect of language can only be observed in positive strategies but not negative strategies. This provides initial support for our hypothesis that the difference between language groups is mediated by reliance.

To further test our hypothesis, we conducted a Bayesian mixed-effects mediation analysis to examine whether reliance mediated the effect of AI politeness on ΔPositive and ΔNegative. ID and scenario are also treated as random effects. While the primary analyses in this paper used null-hypothesis significance testing, Bayesian mixed-effects approach was used for the mediation analysis due to the limitations of standard mediation packages in handling multilevel models with random effects. We conducted two Bayesian mixed-effects mediation analyses to test whether participants’ reliance on AI mediated the effect of AI politeness on ΔPositive and ΔNegative, respectively. The results are presented in Table 1.

The results first indicate that AI politeness has strong direct effects on both positive and negative politeness strategies, with no evidence of mediation through reliance. This means that simply being exposed to a certain type of AI suggestion (positive vs. negative) is enough to shape participants’ writing. On the other hand, the effect of language is mediated by reliance. Specifically, for positive strategies, language has marginally significant influence on ΔPositive both directly and indirectly, and reliance partially mediates the effect. For ΔNegative, the effect of language is primarily indirect, suggesting that differential changes in politeness strategies are not fully due to language background but because NNES leans on AI more often.

4.2.1 Effect of AI politeness and language on Social Perception. Analysis of the number of politeness strategies included in the emails confirmed that the written content is influenced by AI politeness and the effect is different across language groups. Given the theoretical grounding that politeness strategies used are linked to social perception on three dimensions, distance, power and imposition, we further tested H2 to see if AI politeness affects social perception. Results partially confirmed this hypothesis, indicating an effect of AI politeness on change in distance perception, but no effect was found on the power and imposition dimension. Additionally, we analyzed if participants in different language groups are impacted differently (H3b). This hypothesis is not confirmed, indicating that NNES and NES are not influenced differently by AI suggestions in terms of social perception.

In the analysis, we also used a mixed-effects model with AI politeness and language as fixed effects, treating scenarios and individual participants as random effects. The outcome variables were changes in perception (ΔD, ΔP, ΔR), calculated as the difference between post-writing perception (after using AI) and pre-writing perception. For each participant, a total of six perception changes were calculated, ΔD, ΔP, ΔR for both negative-AI and positive-AI conditions. Changes in perception was compared across negative-AI and positive-AI, with negative-AI condition and NNES used as reference levels. The results show that the AI politeness had a significant main effect on ΔD (β = −0.61 , SE = 0.25 , t(49.67) = −2.42 , p = 0.02), with negative-AI leading to a larger increase in social distance perception. However, there is no significant effect of AI politeness on ΔP (β = −0.19 , SE = 0.29 , t(101) = −0.65 , p = 0.51) or ΔR (β = −0.37 , SE = 0.31 , t(50.38) = −1.22 , p = 0.23). The effect of language is not significant on any of the three dimensions, and there is also no significant interaction effect between AI politeness and language.

This result suggests that the ΔD differs across AI politeness conditions. In the negative-AI condition, participants tend to change their distance perception more towards the distant end, compared to the positive-AI condition. NNES and NES are not different in the magnitude of perception change. The overall result for all three dimensions is visualized in Figure 6. This aligns with Brown and Levison's theory connecting use of negative strategies with the perception of more distant relationships, however, the connection to the power and imposition dimension is not supported by our results.

While it confirms AI politeness leads to differences in ΔD, we did further analysis to directly compare post-writing perception (D_post-writing, P_post-writing, R_post-writing) with pre-writing perception (D_pre-writing, P_pre-writing, R_pre-writing) to confirm whether writing with positive-AI and negative-AI changes perception from pre-writing baseline. The result does not provide strong statistical evidence to suggest a significant change. We did this analysis through a mixed effect model that was run on the perception value, instead of perception change, with a new variable time to capture the perception ratings collected pre-writing task vs. post-writing task. On the distance dimension (D_pre-writing vs. D_post-writing), we did not observe an interaction effect between time and AI politeness (β = 0.42 , SE = 0.31 , t(151) = 1.37 , p = 0.17), and when post-writing is set as the reference, there is no main effect of AI politeness (β = −0.30 , SE = 0.22 , t(152) = −1.37 , p = 0.174). This shows that distance perception did not change in different directions after writing with AI tools. This contradicts findings on the effect of AI politeness on ΔD, which calls for more careful interpretation of the results. It can suggest that the effect of the AI politeness on perception is very subtle, and is hard to detect through a one-off experiment like the ones we conducted. Future analysis with a larger sample size and long-term exposure to AI manipulation is needed to find more robust results.

Fig. 6. Social perception comparison before and after writing tasks.

4.2.2 Mediating Effect of Change in Politeness Strategies on Change in Distance Perception. As the previous section suggests an effect of AI politeness on change in politeness strategies and change in perception, we hypothesize that the effect on perception could either come from a direct effect of exposure to AI suggestions, or it could be mediated by the change in politeness strategies in writing (i.e. people's perception changes as a result of the changes in their writing). Through our analysis, we were able to confirm that the change in perception can be attributed to the direct effect of the AI politeness, not mediated by the change in writing.

We conducted a mediation analysis using a mixed-effects model that included random intercepts for scenario and participant ID. AI politeness served as the predictor, and the change in politeness strategies was specified as the mediator. As changes in politeness strategies include both ΔPositive and ΔNegative, we employed changes in relevant strategies as our measurement. When the condition is negative-AI, AI suggestions predominantly include negative strategies, and the ΔNegative is the relevant metric that matters most. Similarly, when the condition is positive-AI, the ΔNegative is used to capture the mediating factor.

Results reveal a significant total effect of AI politeness (β = −0.61, p = 0.02), which is aligned with previous findings. There is a significant direct effect (β = −0.64, p = 0.02) and non-significant in-direct effect (β = 0.03, p = 0.47). This indicates that the effect of AI politeness on ΔD is direct, suggesting that participants’ perceptions shifted simply through exposure to AI suggestions, rather than as a result of changes in their use of politeness strategies during writing.

4.3.1 Effect of AI politeness and language on email quality evaluations. In addition to the effect of AI politeness strategies on participants’ writing strategies and social perception, we tested whether using AI that adopts these strategies might affect how the emails are perceived, especially the writing quality. The results suggest that using AI tools leads to the emails being perceived more negatively, but the effect does not differ across AI politeness and language.

We performed a mixed-effects analysis to examine the main effects of writing conditions (baseline vs. positive-AI vs. negative-AI) and language (NNES vs. NES) on email quality, as well as their interaction, with individual participants and writing scenarios as the random effect. H1 and H2 ask about the differential impact of positive-AI versus negative-AI, so we focus on measurements of change (ΔPositive, ΔNegative, ΔD, ΔP, ΔR). In the analysis of email quality, we focus on the decrease in quality in both conditions, so we study the quality rating itself. Perceived writing quality was measured by asking 4 raters who were blind to the conditions to rate the emails, and the final quality rating is the average across the raters. Details of the measurement is included in Section 3.5.3.

The results revealed a main effect of writing condition on email quality, with significant difference between negative-AI and baseline condition (β = −0.39 , SE = 0.13 , t(100.39) = −3.05 , p = 0.003), and weak statistical evidence suggesting difference between positive-AI and baseline condition (β = −0.22 , SE = 0.13 , t(100.27) = −1.75 , p = 0.08), which indicates a decrease in evaluated email quality after writing with AI. However, the difference between the two AI conditions (negative-AI vs. positive-AI) is not significant (β = −0.17 , SE = 0.13 , t(100.50) = −1.30 , p = 0.20). We also did not find a main effect of language or an interaction effect between writing condition and language, suggesting that NNESs and NESs do not differ in their email quality and NNESs’ email quality is not more impacted by AI suggestions.

4.3.2 The Effect of Baseline Social Perception on the Change in Perception. WWhile previous analyses show that language group does not predict the changes in social perception, we ran an additional analysis to test whether another individual factor — baseline social perception — affects social perception change. Baseline social perception was measured by averaging each participant's pre-writing ratings across 10 scenarios and was used as an indicator of their trait-level tendency to perceive social situations. We are interested in this factor as we hypothesize that people whose baseline perception is closer to the population average may be more susceptible to the influence of AI politeness, as they have less strongly defined views about social situations and thus are more likely to be impacted by AI. However, results suggest a trend opposite to the initial hypothesis, showing a convergence effect in perception. Participants with more extreme baseline social perceptions were more influenced by AI suggestions in terms of perception change, and this effect applies across three dimensions.

The effect of baseline social perception is also tested with a mixed-effects model, with ΔD, ΔP, ΔR as outcome variables, baseline social perception as fixed effect and ID and scenario as random effects. As indicated in Table 2, change in perception is negatively correlated with baseline social perception in all three dimensions. Participants with initial low perception value tend to increase their perception, and those with initial high value tend to decrease their perception. This can lead to their final perception being more homogenized after writing with AI.

As an additional exploratory analysis, the convergence effect we found might suggest that AI reduced individual differences among participants in terms of social perception, leading to more similar views of the social situation. However, the result should also be interpreted with caution, as we are not able to compare changes in perception in the baseline condition where users did not write with AI tools. Thus, we can not rule out the possibility that the convergence is the result of writing. Our results can only suggest preliminary directions that require further investigation.

5.2.1 Effect of AI Politeness on Politeness Strategies in Emails and Social Perception. Results show a significant effect of AI Politeness on politeness strategies in emails, and this also extends to the human-written components. This could be an influence directly applied at the linguistic level, or potentially come from the shaping of social perceptions and then indirectly change writing strategies. While both theories are supported by existing literature and align with aspects of our results, neither fully accounts for the underlying process.

Previous theories and studies have explored the effect of linguistic alignment. Based on the interactive alignment theory [45], during conversations, people would automatically align mental models, which leads to synchronization in linguistic representations. This is supported by many other works that suggest people's linguistic style converges during the establishment of friendships [32] and in online communities like Reddit [3]. In addition to linguistic alignment in human-human interaction, this alignment effect has also been found in human-AI interactions, especially in interaction with conversational agents [55]. We believe that the automatic process of aligning mental models and converging linguistic style might be activated when human writers are exposed to AI suggestions in our study. Our results provide additional evidence to suggest that the linguistic alignment effect also applies to human-AI interactions.

On the other hand, our results also indicate that changes in social perception happen concurrently with changes in writing strategies. This suggests that another potential mechanism behind the change in politeness strategies in writing might be through the change in social perception. However, the mixed findings in our study suggest that while AI politeness might have an impact on social perception, the effect is relatively subtle, and is not the main reason that contributes to changes in politeness strategies used in the emails.

We additionally hypothesize that there might be a bottom-up effect where people's writing changes as a result of using AI assistants, and thus changes their social perception. However, mediation analysis does not support this relationship. Thus, there are reasons to believe that the link between change in writing strategies and change in perception is relatively weak, and AI suggestions impacted change in these two variables through different mechanisms. There is room to explore further the specific reasons behind the change, to help provide more implications for future design and development of AI tools.

5.2.2 Effect of language on Politeness Strategies in Emails. Results found NNESs are more affected by AI politeness in terms of their strategy change in emails. This effect is partly mediated by reliance on AI tools.

This aligns with previous work on the usage of AI writing assistants among NNESs, suggesting that NNESs are generally more reliant on AI tools and believe that these tools are better than their judgments. This reliance is also found to be more prevalent among people with lower proficiency levels than those with higher levels [30].

Additionally, we found the differential effect of language is mostly significant with positive strategies but not negative strategies. We propose two possible explanations for this. Firstly, by definition in the Brown & Levinson theory, negative strategies are considered to be closer to what we consider “conventionally polite”. This means that this could be a relatively more familiar type of politeness expression to many people. Given this assumption, both NESs and NNESs included in the study might have built enough familiarity towards negative politeness strategies, but positive politeness strategies can be more foreign to NNESs because they are less frequently used in general. When presented with such AI suggestions, this lack of familiarity might lead to more re-strategizing in email writing for NNESs. Secondly, the AI suggestions, although carefully fine-tuned, might still include negative strategies in the positive-AI condition. Our manipulation check (Section 3.2.2) shows that people are more likely to mistake negative-AI for default GPT suggestions, while positive-AI is more distinguishably different from default GPT suggestions. With negative strategies present in both positive-AI and negative-AI conditions, the effect of AI politeness might not be as salient, thus not reflected in our statistical test results.

5.3.1 Design Implication 1: Incorporating Social-Context Awareness in AI-Mediated Communication. In our study, the AI writing assistants were calibrated to predominantly incorporate positive or negative strategies. When AI suggestions are misaligned with writers’ initial perceptions of the social situation, it may create distorted perceptions with inappropriate writing strategies. This calls for the design of AI writing assistants that are more aware of the social context, and suggest content suitable for the social scenario.

The relevant social context is both subjective and objective. AI writing assistants can be designed to capture the objective social relationship and provide suggestions that fit the general context of a specific type of writing. For example, IntroAssist [22] is designed specifically to help write introductory requests, where the social relationship is relatively consistent. However, social perception can also be subjective and dynamic, as the relationship is negotiated between members in conversation [12]. Therefore, AI writing assistants should offer suggestions that align with the writer's interpretation of the scenario while ensuring the message is accurately conveyed to the receiver. Previous studies have explored the development of receiver personas and providing tailored feedback based on the intended persona [7]. Additionally, AI assistants can also actively collect the writer's perception in the social scenario, to adapt writing strategies accordingly. For example, asking writers to input their perception of distance, power and imposition can help AI suggest politeness strategies that are better aligned with the writer's evaluation. Politeness is one of the important aspects of AI-mediated communication, while other underexplored aspects might also be relevant. Further studies are needed to explore the link between social context and language use, to create AI suggestions that better align with the writer's intentions.

5.3.2 Design Implication 2: Preserving Writer's Individualized Social Perception and Writing Style. Additional analysis in the study suggests a convergence effect in perception, indicating that AI suggestions lead to more homogenized social perceptions. This homogenizing effect aligns with previous work that suggests AI might homogenize cultural differences [1] or lead to less variation in creative ideations [4]. This finding can have significant implications. While having less diverse social perception may facilitate communication, as the reduction in perception differences might bridge misunderstandings, it could also have negative implications by diminishing individual characteristics.

This suggests design implications for AI writing assistants to help writers preserve their uniqueness in the communication process. This can be addressed by developing AI assistants that mimic the writing styles of writers to minimize discrepancies between AI writing style and human writing style. Also, as the results suggest that AI assistants’ influence on people's perception and writing strategies is largely unconscious, it is important to develop mechanisms to promote awareness among writers and reflect on how they have been influenced by AI. There is potential to leverage visualizations to help writers identify the discrepancies between AI suggestion style and writers’ personal style.

5.3.3 Support for non-native English speakers. As indicated in Section 4.1.3, NNESs’ writing strategies are more influenced by AI writing assistants, and this is mediated by their higher reliance on AI tools. This might be due to the lack of confidence in their ability to write socially appropriate messages and their tendency to treat AI as a tool with authority. Previous studies have suggested that the main challenge NNESs face when interacting with AI assistants is the lack of ability to assess the appropriateness of AI suggestions [30]. There are multiple layers of challenges, including ensuring language accuracy, delivering messages that reflect the writer's intentions, and adopting communication strategies that align with social norms [13]. Thus, to help NNESs better evaluate AI suggestions, AI assistants should focus on ensuring that the suggestions align with the users’ social intentions and are interpreted by the receivers as intended. Previous studies have explored ways to help NNESs navigate the challenges, for example, providing examples to help them evaluate the suggestions in context [30]. We suggest that the previous two design implications – the incorporation of contextual awareness and mechanisms to preserve the writer's personal characteristics – would greatly benefit non-native speakers. By emphasizing the social context, NNESs can gain confidence that AI suggestions are tailored to their specific communication scenarios, which allows them to write messages that accurately convey their intentions. Additionally, mechanisms that preserve the writer's unique characteristics will ensure that NNESs are not overly influenced by AI tools predominantly trained on Western-centric data.

In addition, it is important for AI assistants to empower NNESs and not exacerbate their overreliance on AI tools. While tools that are perfectly tailored to the social context and NNESs’ personal characteristics can be extremely supportive, they may not directly address their feelings of being foreign or inadequate. On top of providing writers with the most appropriate response based on AI's assessment of the situation, AI assistants for non-native speakers need to focus on scaffolding the process, helping them translate the social context to a more familiar context they could understand, and helping them evaluate the accuracy of their responses using reference points that they can understand from their own culture.