Adopting chatGPT within the financial technology classroom

ChatGPT has been the subject of much debate in the academic community, particularly on how to prevent its unauthorized use in classrooms. However, students can benefit from understanding how generative artificial intelligence (AI) can be used as a tool to save time and improve performance on writing assignments.

Craig Hurwitz, executive director in residence at the Pratt School of Engineering, asked graduate students in the Emerging Trends in Financial Technology (Fintech) course to create a first draft of an article with the help of ChatGPT. His working assumption was that when students enter the workforce, they will have access to AI and the ability to use it for productivity purposes. He wanted to try ChatGPT in his course to give students a first-hand look at how generative AI can be used.

For the assignment, the class was instructed to read a case study and each student chose a Fintech approach to help solve a specific challenge mentioned in the case. Their written assignment was a 750-word executive summary that convinces the coach (who plays the Venture Capitalist) why he should consider meeting to discuss a potential investment.

The instructions called for each student to create a first draft using ChatGPT. After creating the draft, they imported the draft into Microsoft Word and turned on Track Changes. They edited the rough draft and submitted the final draft with Track Changes still on.

This gave the professor the ability to see the draft that was generated by ChatGPT, the changes made to that draft, and the final product submitted by each student. The final, unedited production ranged from about 20 percent to over 90 percent. After the task, the students completed a survey about their experiences with the task. This post summarizes student feedback on using ChatGPT.

Artificial intelligence as a business skill

The students viewed this task as an important exercise because they anticipate that they will use AI in the workplace. There have been many comments about how this assignment was a learning opportunity on how to use an AI chatbot effectively and how this will be a valuable skill when they graduate and take on full-time roles in the workforce.

As one student explained, “I really enjoyed working on this task. I think what you allow us to do is what will happen in real life. Completing this task is a good practice for us to use AI to improve our work.”

One of the specific skills students have acquired is how to write effective ChatGPT prompts. Many students realize that developing an appropriate prompt is an iterative process and key to a successful experience. Almost all of the students embarked on this exercise by trying to use a single prompt to create the entire essay.

However, most students found the outputs to be too general and too repetitive in most cases. The most common response to this has been to provide ChatGPT specific follow-up questions using descriptive and concise text designed to generate output for a specific paragraph, or more specifically, a bullet point.

Many students commented on the importance of training a ChatGPT to take on a specific role before having it create any written output. One student commented in a post-task survey: “I would focus more on training the AI ​​to adopt a specific personality, rather than spending a lot of time trying to figure out the right mentor.”

Writing efficiency

Each student claimed that using an AI chatbot to help create a first draft was helpful. Ninety percent of the students said that using an AI chatbot saves time. One student noted, “For every minute you spend in ChatGPT, you save ten minutes, but make sure you give the AI ​​the exact information you want it to work with and that the risk of errors is minimal.”

Twenty-one of the 36 ESL students in the class felt another benefit. Each of them indicated that they already had experience with ChatGPT prior to this task. A common refrain was that using an AI chatbot helped them write as if English was their native language. However, as one student warned, “The grammar the AI ​​used was perfect, but I found the language vague and uncomplicated.” This comment indicates students’ general understanding of the limitations of ChatGPT.

determinants

While students found ChatGPT useful, there was an important comment about the limitations of this generation of technology. Limitations generally fall into one of three categories: 1) incorrect sources, 2) outdated sources, and 3) the output is as good as the prompt. One student remarked “I found that ChatGPT found correct information, but with an incorrect quote”.

More than half of the class found that ChatGPT produced incorrect information and only two out of 36 students expressed confidence in the sources cited by ChatGPT. Students understood that fact-checking is an essential part of using ChatGPT for writing, since source integrity is a major issue. In addition, since the task of the students was to highlight emerging trends in Fintech, they could not rely on outdated ChatGPT data (September 2021).

Note that the students used the GPT-3.5 model to map this AI; This was before the release of GPT-4.0, when the ability to cite sources was somewhat improved, but relying on sources provided by GPT is still a problem.

Several students commented that they enjoyed working on the assignment, but are aware of ethical considerations such as intellectual property, and appreciated the necessity of guardrails. Understanding students using ChatGPT can be just one step in the writing process. “A bot should not be used as a final response, but only as a way to understand a topic or exchange ideas. ChatGPT’s ability to aggregate sources and condense information is great, but it shouldn’t replace our filters and our ability to do the same work ourselves.”

Conclusion

Generative AI capabilities are evolving rapidly. It is a productivity tool that is likely to gain wide acceptance and use in the workforce. Training students how to use technology appropriately and effectively is an important life skill. For those concerned about whether the output generated by ChatGPT is the work of the tool or the requester, there is no substitute for doing basic research, getting a clear understanding of what you want to communicate, and knowing beforehand, the written structure you expect to get from ChatGPT.

From the perspective of a professor who has spent most of his career in industry, it was rewarding to see how quickly students adapted and improved technology, and how this resulted in articles being produced efficiently and, in many cases, of a higher quality than they could have achieved otherwise.