| Pages in topic: < [1 2 3 4 5 6] > | Introducing AI to KudoZ: human expertise and AI combined in support of the most helpful answer Thread poster: Lucia Leszinsky
|
|---|
| Why ChatGPT? | Aug 20, 2023 |
On further reflection, if the ProZ staff wanted to incorporate some kind of MT responder into Kudoz, then ChatGPT is a curious choice.
For one thing, its knowledge of the world stops at September 2021, so any events, or any new language, that are more recent than that do not exist as far as it's concerned.
Secondly, if you're going to link Kudoz to a large language model AI, why not go for GPT4, which is supposed to be much better than the version 3 used by ChatGPT? (I ... See more On further reflection, if the ProZ staff wanted to incorporate some kind of MT responder into Kudoz, then ChatGPT is a curious choice.
For one thing, its knowledge of the world stops at September 2021, so any events, or any new language, that are more recent than that do not exist as far as it's concerned.
Secondly, if you're going to link Kudoz to a large language model AI, why not go for GPT4, which is supposed to be much better than the version 3 used by ChatGPT? (I have no personal experience of it, but just from reading reviews.)
More generally, translation is not the main purpose of these AIs, and there are other MT engines that might be better suited to a translators' website.
Perhaps Lucia, or another staff member, could explain the rationale behind this decision.
Or was it just an idea somebody came up with one night at the pub, and everyone thought it sounded cool? ▲ Collapse
| | | | Samuel Murray 
Netherlands
Local time: 19:30
Member (2006)
English to Afrikaans
+ ...
| And the bot's confidence is set to "5" | Aug 20, 2023 |
Barbara Carrara wrote:
Not only is the answer preceded by 'A translation would be', which I guess will become a leit motiv with this 'user', but it is extremely long. Is ChatGPT allowed more space than us?
And to add insult to injury: the bot gave itself a confidence rating of "5".
When humans answer, a high confidence rating usually means that the person answering is highly skilled in this field or in this language and that they are personally vouching for and staking their reputation on the answer. Since ChatGPT is or does or has none of these things, it should not give itself a confidence rating of "5". How does ProZ.com determine how confident the bot it?
| | | |
Support the machines? In my naïveté, I thought it was the other way around and machines were created to support us mortals. Science fiction movies have long been presaging a world governed by robots. Are these fictions now on the brink of becoming reality?
| | | |
Samuel Murray wrote:
How does ProZ.com determine how confident the bot it?
The answer is in the FAQ: https://www2.proz.com/faq/246430#246430
1.35 - How are AI KudoZ answers generated?
In order to generate a KudoZ answer, we send ChatGPT three different prompts: one to get a proposed translation of the source term taking into account the information supplied by the asker (languages, fields, contect), one to obtain an explanation of the proposed translation that also includes sample sentences, and a last one to get confidence level information (i.e. to know how confident the AI feels about its proposed translation). Once this information is returned by the AI, a KudoZ answer is automatically built with it and posted together with the question (provided the asker has selected the AI-generated answer option).
IOW, the confidence is determined by the bot itself. And as we have seen in many examples of ChatGPT output (and not only in Kudoz...) the bot is always supremely confident that it has delivered a top-notch answer - until it gets another prompt telling it the answer is utter nonsense.
| | |
|
|
|
Becca Resnik
United States
Local time: 13:30
Member
German to English
+ ...
IrinaN wrote:
Plus, younger generations love..."hard manual labor" of thinking and researching before writing - much less than we do
@IrinaN: I am compelled to say that the statement above is a grossly inaccurate overgeneralization. An inquisitive human mind is irrespective of time.
As for "AI" and KudoZ in general:
- I will be among those not renewing my membership should this trajectory continue and/or expand.
- These "AI" models are not even appropriate/intended for MT.
- It is deplorable and even insulting that the ProZ budget is going to this development rather than the numerous other important upgrades that have been proposed over the years (even since long before I joined).
- The point of KudoZ is to request help when a translator is stuck in his or her research. ChatGPT and the like, if the translator wants to use these tools, needs to have been checked first – just like any other form of independent online research.
- What is the response to the issue that users' attempts to improve rankings in search results will be decimated? This isn't a feature I participate in but is an entirely valid point.
These rebuttals have all been made before, and I have many other thoughts on the topic – this development is inherently flawed in many ways. However, I find it important to at least be another voice making these statements in particular.
| | | | Mr. Satan (X)
English to Indonesian
All the issues mentioned in this thread could've been ironed out during the beta-testing phase. I was meaning to volunteer myself, but last week I was rather preoccupied with work. Your beta-testing window is way too short, IMHO. Only eight days. Aside from Frost and Beijer, I can't tell if there were more people involved in testing this new feature before officially rolling it out to the public. While I'm sure the two gentlemen I mentioned have the technical expertise to provide valuable feedba... See more All the issues mentioned in this thread could've been ironed out during the beta-testing phase. I was meaning to volunteer myself, but last week I was rather preoccupied with work. Your beta-testing window is way too short, IMHO. Only eight days. Aside from Frost and Beijer, I can't tell if there were more people involved in testing this new feature before officially rolling it out to the public. While I'm sure the two gentlemen I mentioned have the technical expertise to provide valuable feedback, this is still not optimal.
I also would add that ProZ.com's implementation of ChatGPT is really weird. Why does the AI need its own dedicated account? Is there any particular reason this feature can't be affixed to the website in a more seamless manner? And I know the points it accumulates don't count in the directory ranking. But why granting it points since the first place?
[Edited at 2023-08-21 00:32 GMT] ▲ Collapse
| | | |
Jennifer Levey wrote: The answer is in the FAQ: https://www2.proz.com/faq/246430#246430
In order to generate a KudoZ answer, we send ChatGPT three different prompts: ... and a last one to get confidence level information (i.e. to know how confident the AI feels about its proposed translation).
Here is the big fallacy. The AI does not "feel" anything at all. It is incapable of feeling; it is incapable of judgement, in the sense we normally use the word.
It's just incredibly sophisticated algorithms working on massive amounts of data. The rest is fudge and fakery.
That doesn't mean it's not useful. I use ChatGPT on a regular basis, as a kind of not very bright research assistant.
But it's important to be aware exactly what we're dealing with here. The AI cannot "feel confident" about its proposed answer and it has no concept of accountability, should it get things completely wrong.
Elsewhere in the FAQ there is this question:
If AI technology is available out there and language professionals are using it (in many different ways), why not include it in KudoZ and enable them to realize their full potential?
First, this suggests that language professionals cannot "realize their full potential" without using AI technology, which I find insulting. Second, it begs the obvious answer that KudoZ is not the place for this kind of thing.
The more I think about it, the more I am of the opinion that this whole idea is a crock and should be abandoned forthwith.
| | | | | Translation example | Aug 21, 2023 |
Barbara Carrara wrote: Ah, yes, you've obtained a 'very quick' answer from it, but is it the correct one, though? https://tinyurl.com/2y93v3dz I don't speak Chinese, but considering that you had to adjust its answer in the 'Note to answerer' window, therefore wasting your time to help the AI, its 'performance' doesn't strike me as very successful.
I didn't check out the example before, but now I have done so. I suggest that others should do the same, especially those who are still undecided about this addition.
Because what Barbara didn't mention is that, although I'm not able to judge its literal accuracy, ChatGPT's English translation here is barely comprehensible gibberish.
In fact, this example provides powerful evidence that this is a terrible idea.
| | |
|
|
|
Jo Macdonald
Spain
Local time: 19:30
Italian to English
+ ...
| Chat bot's "top confidence" ¿translation? on Proz, NOW! | Aug 21, 2023 |
"A century of cultivation brings forth the same boat to cross the sea; a thousand years of cultivation brings forth together peaceful sleep in the same bed."
And you asked it three times...
Thanks for not including this rubbish in the old interface, please don't.
2 KudoZ points were awarded for this answer
Chat bot earning points fast, Proz losing points fast... Top translators' platform helps convince non-native clients gibberish now "acceptable".
Philip Lees wrote: Barbara Carrara wrote: Ah, yes, you've obtained a 'very quick' answer from it, but is it the correct one, though? https://tinyurl.com/2y93v3dz I don't speak Chinese, but considering that you had to adjust its answer in the 'Note to answerer' window, therefore wasting your time to help the AI, its 'performance' doesn't strike me as very successful. I didn't check out the example before, but now I have done so. I suggest that others should do the same, especially those who are still undecided about this addition. Because what Barbara didn't mention is that, although I'm not able to judge its literal accuracy, ChatGPT's English translation here is barely comprehensible gibberish. In fact, this example provides powerful evidence that this is a terrible idea.
[Edited at 2023-08-21 08:00 GMT]
| | | | expressisverbis
Portugal
Local time: 18:30
Member (2015)
English to Portuguese
+ ...
| Just out of curiosity... | Aug 21, 2023 |
Just out of curiosity, I posted a question on Kudoz and found the option in the new version.
I wanted to try it out and, at the same time, I wanted to know what would be the solution provided by AI... well, it was funny, it made me laugh of course.
It's nice to not have this option in the old version, and I hope we can still enjoy and use the old interface.
Regardless of whether Artificial Intelligence prevails and sticks around, my thanks and points will be awarded to a transl... See more Just out of curiosity, I posted a question on Kudoz and found the option in the new version.
I wanted to try it out and, at the same time, I wanted to know what would be the solution provided by AI... well, it was funny, it made me laugh of course.
It's nice to not have this option in the old version, and I hope we can still enjoy and use the old interface.
Regardless of whether Artificial Intelligence prevails and sticks around, my thanks and points will be awarded to a translator with Real Intelligence, by which I mean someone who gives me an answer in a highly professional way.
In the case of a completely literal and nonsensical answer, then I prefer the machine... we all know that there are members on Kudoz who don't even know the meaning of translation and how to be a professional translator. ▲ Collapse
| | | | | Clarification | Aug 21, 2023 |
Mr. Satan wrote:
Aside from Frost and Beijer, I can't tell if there were more people involved in testing this new feature before officially rolling it out to the public. While I'm sure the two gentlemen I mentioned have the technical expertise to provide valuable feedback, this is still not optimal.
Just for the record: nobody from Proz has contacted me in relation to testing. As I neither use nor like the new user interface (which is an excellent example of the adage ‘if it ain't broke, don't fix it’), it would have been a bit awkward anyway. An AI option is not going to make me give in to Proz's odd crusade to switch to an inferior UI.
| | | | Mr. Satan (X)
English to Indonesian
Thomas T. Frost wrote:
Just for the record: nobody from Proz has contacted me in relation to testing. As I neither use nor like the new user interface (which is an excellent example of the adage ‘if it ain't broke, don't fix it’), it would have been a bit awkward anyway. An AI option is not going to make me give in to Proz's odd crusade to switch to an inferior UI.
My sincerest apologies, and thank you for clarifying. Based on Lucia's post in the beta-testing thread, I had assumed that you were also involved in testing this new feature. Though I know Beijer did test it. I'm curious of what the response was to the issue he brought up.
| | |
|
|
|
Zea_Mays
Italy
Local time: 19:30
Member (2009)
English to German
+ ...
| GPT says the opposite when asked directly | Aug 22, 2023 |
I answered a Kudoz question where an AI generated answer was posted,
and out of curiosity asked GPT the exact same question directly,
and it answered with exactly the opposite of what it had previously suggested in Kudoz (with CL 4).
Just to tell how reliable the bot is.
| | | | | Stubborn ProZ's AI Policy | Aug 22, 2023 |
As Barbara rightly said before, "this nonsense [...] once again proves how ProZ is not on our side."
I would simply add: How ProZ Staff, confronted by such a fierce opposition tellingly displayed by their own subscribers, can stubbornly pursue their foolish policy by implementing AI in their system by hook or by crook, whereby digging their own grave and ours by the same token?...
As some of us have just warned it down here, I also consider to question the renewal issue... See more As Barbara rightly said before, "this nonsense [...] once again proves how ProZ is not on our side."
I would simply add: How ProZ Staff, confronted by such a fierce opposition tellingly displayed by their own subscribers, can stubbornly pursue their foolish policy by implementing AI in their system by hook or by crook, whereby digging their own grave and ours by the same token?...
As some of us have just warned it down here, I also consider to question the renewal issue of my ProZ membership subscription in due course as well...
I'm not willing to fatly pay a Service Provider to have them me hung high and dry!
[Modifié le 2023-08-23 05:32 GMT] ▲ Collapse
| | | | Zea_Mays
Italy
Local time: 19:30
Member (2009)
English to German
+ ...
| No alert on AI answers | Aug 22, 2023 |
I just noticed that there is no statement like "This is an AI generated answer".
| | | | | Pages in topic: < [1 2 3 4 5 6] > | To report site rules violations or get help, contact a site moderator: You can also contact site staff by submitting a support request » Introducing AI to KudoZ: human expertise and AI combined in support of the most helpful answer | CafeTran Espresso | You've never met a CAT tool this clever!
Translate faster & easier, using a sophisticated CAT tool built by a translator / developer.
Accept jobs from clients who use Trados, MemoQ, Wordfast & major CAT tools.
Download and start using CafeTran Espresso -- for free
Buy now! » |
| | Anycount & Translation Office 3000 | Translation Office 3000
Translation Office 3000 is an advanced accounting tool for freelance translators and small agencies. TO3000 easily and seamlessly integrates with the business life of professional freelance translators.
More info » |
|
| | | | X Sign in to your ProZ.com account... | | | | | |
Login to reply/comment 
