Overview
When ChatGPT launched, discussion about AI flooded the internet. Some people were optimistic about what it could do for education. Others worried about the problems that fast AI adoption could bring. Its effect on people learning to program became a central topic in tech.
We combined netnography with a focus group. We found that, used the right way, ChatGPT can make learning to program much more efficient. But many people remain sceptical about AI in a learning environment. Participants also shared how they see ChatGPT being used in industry.
Background
People increasingly use ChatGPT in daily life, and questions about its role in education keep growing: essay writing, standardised tests, language learning, coding and more. Some praise it for personalised learning. Others doubt what it will do to the education system. Either way, the technology is here to stay, and students and educators are adapting.
ChatGPT’s ability to generate code quickly in common languages has impressed many students and programmers. Using it to help with coding and to learn new languages has become popular among students and people entering software engineering. At the same time, some programmers fear their jobs could become redundant, unlike jobs such as journalism or nursing that depend heavily on human factors.
We wanted to understand how ChatGPT affects how students learn programming, and how people entering software engineering feel about it. They are the ones who will shape the future of the field.
Research question
How does ChatGPT impact students’ learning of programming?
Sub-questions
- How do students trust the information from ChatGPT?
- How do students use ChatGPT to get the best answer?
- How do students feel about using ChatGPT to learn programming?
Participants
The netnography phase looked at online discussion of ChatGPT and AI in education and learning. That data came mostly from educators, industry professionals, students and current learners.
The focus group phase narrowed to software engineering (SWE) education. We recruited current students and recent graduates who had taken at least two programming classes and had used ChatGPT while learning.
Method
- 60
- Posts collected, 20 each from YouTube, LinkedIn and Reddit
- 3
- Participants in the focus group, all students or recent graduates
- 45–60min
- One session, on Zoom in mid-May
We ran the research in two phases: netnography, then a focus group.
Phase 1: Netnography
We wanted a broad view of students’ experiences and opinions of ChatGPT for learning. We chose YouTube, LinkedIn and Reddit because they host active, rich discussions. We collected 20 text posts from each site, 60 in total.
Each post had to meet these criteria:
- Content: the impact of ChatGPT on students’ learning and their engagement in learning environments
- Keywords: #LearnAI #LearnChatGPT #ChatGPTStudentLearning #AIStudentLearning #ChatbotsInEducation #AIAssistedLearning #AIEducation
- Engagement: at least 5–10 comments or 50 views
We logged each post in Google Sheets, noting experiences with ChatGPT for learning, opinions, and concerns about its impact on learning and critical thinking.
Analysis. We used affinity mapping in FigJam. Each of us coded the posts from one platform. We then turned all 60 posts into sticky notes together and grouped them by code. The most common themes were efficiency, personalisation, critical thinking, trust and accuracy, accessibility, and replacement.
Ethics. The posts were public, but quoting them in academic work could still affect the authors. We cite only the platform, never the poster’s real or pseudonymous name.
Limitations. Each platform’s audience and purpose shaped the tone of the data:
- YouTube comments came mostly from subscribers engaging with creators they liked, so they were largely supportive.
- LinkedIn’s focus on professional identity kept comments respectful, positive and formal. Posts and comments leaned towards optimism and promoting the tool.
- Reddit’s anonymity let people speak more freely, so discussion was more casual and argumentative.
Phase 2: Focus group
We ran one focus group. A group discussion lets participants build on each other’s ideas, which suits a topic as disputed as ChatGPT. The three participants were undergraduate or graduate students, or recent graduates.
Participant criteria
- College student or recent graduate
- Has used ChatGPT to learn programming
- Has taken at least two programming courses
- Willing to join an online discussion in May
Recruitment
- We posted a recruitment message on educational Slack channels and forums, such as university Slack.
- A screening survey confirmed participants met the criteria.
- Participants received an information sheet and consent form before the study.
The session ran on Zoom in mid-May, with one moderator and two notetakers. We used a semi-structured interview guide and recorded the audio for transcription.
Analysis. We transcribed the recording and used thematic analysis. We coded the data, grouped similar ideas, and built overarching themes from the codes. Participant quotes illustrate each theme.
Ethics. Participation was voluntary and participants could withdraw at any time. All data was kept confidential and participants were given pseudonyms. The data was used only for this study, and no presentation discloses identifying information.
Limitations.
- Possible bias. Three participants may not represent the wider population of people learning software engineering. Their unique views could bias the findings.
- Limited time. The session was capped at 45–60 minutes. Some topics may not have been fully explored.
- Specific context. The findings should be read in light of this study design and these participants.
Key themes
ChatGPT usage and benefits
Trust depends on the task. Participants trust ChatGPT for productivity and simple tasks, but not for complex problems or creative work. They were confident using it to summarise articles or write simple content, and much less so for actual coding, design work and harder tasks.
“I’d probably give it a 4 for debugging and productivity during development. I think there’s a lot of aspects where it can be alleviated in the office. But as for actual coding, I’d give it a slightly lower score, like a 3.” — P1
“I’ve used it on design stuff. So it was a project. When I was asked to design a simple cloud-based data pipeline on the AWS environment and instead of reading up on all the documentations and exploring all the different AWS services. I just kind of told ChatGPT the spec that asks it which services I should use, and how I should integrate them, and that gave me a pretty workable solution.” — P2
“Not using it for system design, I wouldn’t trust that much with that. Maybe only using it for general understanding.” — P2
Personalised problem-solving is its biggest advantage. People learning to program often use it for tedious work like debugging and syntax. Focus group participants found it useful for learning new languages and solving problems. They stressed that you need to give it enough context and specific cues to get relevant answers.
“I’m not super familiar with how to prompt directly. Many people are really passionate about it and play around with writing the prompts. But I never really invest time in that. So I feel like the stuff I type in and the stuff I get out, in terms of code quality, isn’t what I need or expect. So I think that’s why it doesn’t help me that much.” — P1
“I started learning python like 3 weeks ago and recently tried out ChatGPT for fun but then realized how good it actually is. I often give it my codes and it points out the flaws and ways I could improve my codes. It literally is like a tutor.” — YouTube
“I find myself curating code more than writing it myself. Copilot is a real timesaver for many trivial tasks.” — LinkedIn
“If you keep it specific enough or direct it to the problem, that’s a good thing as ChatGPT works.” — P3
“I feel like there’s a pretty steep learning curve of getting comfortable in looking for the information you need. And you might not be sure whether these answers are correct or not. But with ChatGPT, you can tune it to explain things in simple terms. And it’s very tailored to the specific problems you face if you provide it with another context.” — P1
It is efficient for high-level work like documentation and summaries. Participants still preferred to write their own code for school projects. They found ChatGPT most useful for manual work such as reading and writing.
“In the past semester, I’ve been doing some research work and I feel like what ChatGPT is really good at doing is summarizing the content and presenting a lot of interesting insights.” — P1
“I still remember in one of our projects meetings when I just type that into ChatGPT and it just cut half the meeting time down because they didn’t have to mindlessly scroll. I thought I was pretty helpful.” — P1
“It’s sometimes a pain in the ass to go to stack overflow and try and solve your problem. If you have Chat GPT, it can not only help you solve your problem for your case. It can also give you a generalized answer and help you learn in a process. What did you miss? And how can you apply that to future work, I think that’s the valuable gap it can bridge as opposed to the workflow of today, just searching on the web and then looking for other people’s experience with regard to your buck, that may not be exactly the same as yours.” — P3
Limitations and concerns
Data leaks and ownership worry professionals. Participants raised concerns about leaking company information and about copyright. They do not base their work solely on generated content, partly because chatbot output may carry bias. They also worried that ChatGPT logs conversations and may use them for training. These concerns point to a need for clear guidelines and safeguards when using ChatGPT at work.
“I don’t trust it yet as much as to base my work off of what it generates. And on a subtle level I also don’t want to bias whatever comes out of me to come from a Chatbot.” — P3
“I can’t use it for proprietary reasons, because ChatGPT is explicitly known for the ChatGPT logs, so whatever conversation you have, and it can be used for its own training purposes. And, as far as code knowledge, and the further it goes, we wouldn’t want that happening for a company, especially when the model itself is not in-house. Plus, I think it’s also a mixture of mistrust” — P3
Sentiment is positive, but over-reliance can hurt learning. ChatGPT can be a double-edged sword. For a beginner learning a new language, it cuts down the endless Googling and dense reading, which lowers the early learning curve. But one participant said heavy reliance could hold back students who are starting to write their own code. YouTube and Reddit commenters shared the same worry. The struggle is part of how students learn. Removing it can stop them from fully understanding the code.
“I think, whenever we’re learning a new language. There’s always a phase where you’ll be Googling everything to figure out. ChatGPT is good for that.” — P2
“I know lots of people, like freshmen and sophomores specifically, using it for learning. A lot of times when they ask me for help, I do tell them to use ChatGPT. Just because a lot of times I think ChatGPT does a pretty good job at explaining stuff. Not for the purpose of getting answers, but to explain why certain things are like this.” — P1
“Students, even college students miss the point of education, education is not about knowing stuff, that’s a side effect, your brain needs to be exposed to the process of thinking in order to develop, just like how neuroplasticity works, if you don’t use that part of the brain you lose it.” — Reddit
“Making a daily effort to review what I’ve previously read has given me a rich long-term memory, which I can compare all newly acquired knowledge with. This has enriched every experience I’ve ever had. Relying on technologies to spoon-feed us knowledge when we need it to solve a problem will probably encourage people to become dumber.” — YouTube
“Relying on ChatGPT sometimes could kill a lot of the learning aspect. We’re in a more of a school setting, so that takes a lot away, and I think Chat GPT has its brilliant movements in which it could be pretty powerful, but I don’t want to over rely on it too much as well.” — P1
Impacts on education and industry
ChatGPT makes learning more accessible. Many praised it as beginner-friendly because it uses casual, easy-to-follow language. Its basic tier is free for most people, so it reaches learners who lack access to traditional education.
“I’m someone from a totally different career path who just decided to learn to code to help improve my own work. I’ve made this decision because AI is giving me the sense that coding is something more achievable for me now (it feels less cryptic when I have someone/something to guide me step by step).” — Reddit
“I’ve been using it for 1 week. It’s so good. It could replace 99% of our teachers. It answers exactly what you want. and gives simple and good explanations. It’s 1000 times better than our uni teachers. Our teachers literally read powerpoints without even explaining anything.” — YouTube
“When I first tested ChatGPT, it blew my mind with awesome results. As an ed-tech consultant, I am always interested in how technology can improve and make education accessible to all. If AI is used correctly, it can significantly improve quality and accessibility to education.” — LinkedIn
Software engineers will still matter. ChatGPT’s skill at teaching people to code sparked many worries about AI replacing software jobs. Discussion often focused on replacement, yet overall sentiment was mostly optimistic. Many people trust engineers’ drive to innovate and their problem-solving skills.
“There is, and always will be, infinite demand for software engineering resources, as projects inevitably become infinitely more complex. AI will, however, replace the need for most entry and mid-level programmers.” — Reddit
“I think of AI and programmers as Wordpress and Web developers. They both have needs and purposes for different uses but both are needed! We can definitely benefit from AI, but it’s definitely not going to take over!” — YouTube
“I believe LLMs are better thought of as virtual assistants, rather than human replacement. In the particular field of education, it is to aid teachers and students to improve learning, hence it shouldn’t be blocked.” — LinkedIn
Insights summary
Online communities and our focus group shared similar views.
- Efficiency and tailored answers are the biggest wins. ChatGPT speeds up learning and adapts material and feedback to each student, which traditional education struggles to do. Many use it to summarise design documents and as a coding co-pilot. It saves time for both students and educators.
- The main fear is a shortcut that undermines learning. As learning and knowledge become more automated, people worry it will weaken proper learning and critical thinking. One participant described an ideal workflow: use AI for tedious tasks like summaries and documentation, and tackle hard, complex problems yourself.
- Lower cost and an easier learning curve could widen access. This could open programming education to underprivileged learners and improve the learning journey through efficiency and personalisation.
- Guardrails can ease the transition. Usage guidelines and workshops could address early concerns and help sceptics accept the technology in education.
Implications
ChatGPT is becoming common in learning and schoolwork. Despite concerns about its effect on students, its use looks inevitable. We hope these findings help educators and students.
- Rethink assignments. A common concern is that using ChatGPT to write assignments saps motivation and blocks critical thinking. Educators could redesign assignments to limit this and better assess learning. For example, instructors could ask students to show problem-solving, critical thinking and communication through group discussions, presentations or other interactive activities.
- Guide use rather than ban it. ChatGPT is a powerful learning tool, but over-reliance threatens learning and creativity. Banning it is hard. Schools could instead set policies for its use and offer training and support, so students use it reasonably, responsibly and ethically.
Future work
- Understand different users. Future studies could look at the needs and preferences of different users, including designers, across ChatGPT use cases and workflows. That could shape guidelines and features for each group, and involving designers could help ChatGPT better support creative work and usability.
- Inform policy. Research is needed to guide policy and governance for ChatGPT across domains, covering legal and ethical issues and accountability. Researchers, policymakers and industry experts could work together on policies that promote responsible, ethical use.
Reflection
Lessons from moderating
Running a good, unbiased focus group takes care. Moderating takes practice, and it helps to account for participants’ personalities and ways of speaking.
- Encourage open discussion. Participants felt limited by the questions and worried that starting a discussion would slow the group down. One wanted to probe another’s opinion but held back because time was nearly up.
- Consider skipping background introductions. Some participants felt their opinions mattered less because they had less experience than others.
- Be ready for changes. One participant arrived late and the others had to wait. We still finished on time, but we should have had something prepared for the people waiting.
Personal reflection
This project let me dig into how ChatGPT affects software engineers, and to reflect on my own research skills.
The biggest skill I built was running focus groups: facilitating conversation, probing for deeper insight and managing group dynamics. These were key to drawing out rich qualitative data on how participants see ChatGPT in software engineering.
Next, I want to run broader and more diverse studies. That means including software engineers at different levels and from different backgrounds, plus industry professionals and other stakeholders. I also want to sharpen my qualitative analysis and explore adding quantitative methods alongside it. And I will be more vigilant about ethics: confidentiality, informed consent and participants’ well-being, while keeping the research valid.