Adversaria

AI Detectors : Bane or Boon?

AI Detector Image

AI detectors are services that can detect whether a given text is AI generated or not. These are typically used for text, however there are a few out there for image/art purposes. For this article, Iā€™ll specifically focus on text-based detectors, however most of the facts should apply on AI detectors of all kinds.

Some of the most well-known ones are Originality.ai, Turnitin, Sapling, etc. Even Grammarly and Quillbot now have their own AI checker.

Why even worry about AI-generated text?

There are quite a few valid reasons why. Some good ones pointed out by Phronesis2000 on Reddit are:

So, how accurate are they?

Unfortunately, most AI detectors, if not all, are very unreliable. Many AI detector will state this in their websites as wellā€¦

ā€¦or atleast the honest ones will.

Mic-Testing!

To demonstrate their reliability, hereā€™s the result of me putting a short passage I wrote myself through several detectors:

Thing is, AIs are not just AIs. They are not intelligent. Theyā€™re merely Language Learning Models that have been taught through human writing. Imagine feeding a baby healthy food and junk food, and asking them to tell which oneā€™s which. For many foods itā€™ll be easy to tell, but what about well-made healthy foods, or herbal-tasting junk food?

The line gets even more blurry when you consider that you can use AI to avoid some AI detectors. Just go to any AI and request them to rewrite the generated text to avoid AI detectors a few times. I used Gemini and ChatGPT for this, and here are the scores:

While they were mostly accurate, the fact that many of them do flag my writing as AI is concerning, as well as how easy some are to evade.

Another key part that easily gets ignored, is that it isnā€™t effective for certain formats and topics. The text I used was on the topic of Economics, and hence I had to use specific terms, making the text somewhat repetitive. Certain formats donā€™t lend well to AI detection as well, resulting in false positives and negatives. Additionally, use of grammar correction tools like Grammarly can often result in being flagged for AI as well.

Here are some credible sources on the reliability of AI Checker Tools:

This answer on reddit has even more links to credible sources, if needed.

A dangerous precedent

These services alone wouldnā€™t be all that problematic. However, they are now widely being used in schools, colleges, and businesses as a way to judge their candidates. Searching at any Internshala, Naukri.com, or similar job boards will net you a host of postings stating ā€œNO AIā€ in their descriptions, and each of them uses these services to judge the eligibility of the applicants.

Needless to say, many students have experienced this, and many other freelance writers as well. There are frequent posts made by people who had their assignments and content rejected on r/freelanceWriters and r/chatgpt. Iā€™ve also had an interviewer reject me on the judgement of these services (and I used those very same articles today to judge the accuracy of these services). This is a very dangerous precedent, especially with their unreliability.

There are stories floating around, of excerpts from the Bible or the Constitution being flagged as AI-generated. It absolutely makes sense too, since all these AI detectors are AI themselves, and those texts were most likely used as learning material for the AIs as well.

Other ways to verify

If AI Detectors cannot be trusted, then what options do we have?

Quite a few.

One of the common suggestions Iā€™ve seen being recommended is to use Google Docs. Google Docs retains project history, which can then be replayed with extensions like Brisk. This lets you see how the content was created (copy pasted from somewhere), or typed out by hand with corrections. A similar idea is to ask the writer to submit drafts/outlines/revisions alongside the finished product. Both have its own flaws, but checking if the drafts are consistent with the submitted texts, etc. can be more helpful than a mere AI detector.

One can also ask them about their writing process, about how they created the text, what resources they used, what they already knew or what they learnt about the topic, etc. to know if AI was used or not. This, alongside fact-checking (as AI-generated content has quite a few inaccuracies), is a good way to spot AI-users.

However, it is more important to build trust than to spend every moment doubting whether their work was AI generated or not. Perhaps AI detectors will get more advanced later, and be able to weed out AI-text from humans. But there always will be people whose writings will be flagged as AI, especially when AI-generated text is so prevalent in everyday life, because humans learn from things around them. Trust building will go farther than anything else in this changing world today.

Should AI even be used in writing?

Current Scenario:

This is an extremely controversial topic, especially in art communities (including literary circles, as writing is also an art). Even with my deep research and exploration on the topic, I still feel like my knowledge on the subject matter is too little to form a justifiable opinion. However, I firmly believe AI can be used for art in a positive way, like for generating prompts, to find references, to find sources (however, one still needs to fact-check), etc.

Yet the same time, the way AI is being used right now is not in any way beneficial for the greater scheme of things. AI is now being used in google searches, which has ruined all chances of finding accurate information on google. (To be fair, google searches were already starting to worsen in quality by people trying to game the search algorithm.) AI text is also being used for landing page copy, blog posts, and several other things. Even newspapers and outlets are using AI-text instead of human writers, and this has resulted in many inaccuracies.

This has also made the freelance writing market much, much harder to enter. For many freelancers, the smaller writing jobs were a great way to earn some pocket money while also being able to gain experience and learn a more structured writing style suited for better-paying jobs. However, the AI boom has both surrounded us with soulless words, and has left those beginners stranded, with little places to improve their writing. Why would businesses hire an unpaid intern for writing, when they can use AI and churn out more words in less time? But this will overall reduce the number of writers out there, especially for jobs that require better, more specialised writing skill, because there wonā€™t be as many people entering the market.

This all is ignoring the devastating impact AI has on the environment and on climate change, which is a whole ā€˜nother ballpark.

Ideal use:

They can still be used for good. Some ways I mentioned above, like for generating prompts, finding references, etc. There are other uses as well. Discussions on writing forums will easily provide some ways to use AI in your written works, like generating optimised titles and headings. Many also use it as a thesaurus, to find synonyms for words or to find ways to rewrite a sentence that feels off to them.

A sufficient compromise, at least for the above-mentioned problem in writing, is to use human writers alongside AI. AI text is often soulless, and lacks emotions, which is easily remedied by having a human edit the text (or perhaps use that text as reference to create an entirely new essay). Similar jobs exist in many fields, like transcription, where AI transcription is proofread by humans.

Final verdict?

Is there even a need to state one?

Tl;dr: AI detectors, while a very useful idea, are still highly unreliable and should not be used to judge students and candidates. There are other ways to confirm whether AI has been used, like asking for project history or project drafts. Still, trust is an important factor, and building that trust will be way more beneficial than checking for AI content every time.

AI content is still highly controversial. There are many good ways to use it, however currently it is not being used in a way that helps the grand scheme of things, but instead is corroding the entire system. The ideal outcome of this AI boom will be to settle into a compromise between AI and human writers, however it is yet nowhere to be seen.

#AI #AI Detectors #Featured