Finally got some time to sit down and try this thing. It's a good start.
I'm somewhat concerned about some of these value cards to be able to mislead people. I was presented with a choice between the values of "Truth-Seeking" vs. "Accuracy" or something of the like.
At a glance, it looked like the latter made more sense, since it described someone getting overwhelmed with information and then choosing to reduce the cognitive load by picking reliable sources based on reputation and trustworthiness. But looking more closely at the values as articulated, that's not something I'd support in the general case, since it's a shortcut for actually thinking about problems. Sometimes you have to take a shortcut for practical purposes, but it's not a complete substitute for thinking deeply about a problem.
If I had rushed through the tool, I probably would have endorsed the latter. Thinking about it more carefully, though, I endorsed the former instead, since I think it generalizes better.
How are you folks preventing the LLM from ending up with values exactly like this: ones that sound great at a glance, but end up being not what people want when they sit down and think about it more deeply?
In the last step, users deliberate about whether a value is a more comprehensive version of another. This should address what you’re describing, and so far we’ve found that even paid participants give very thoughtful answers there.
Looks like there are only a few dozen respondents so far. How are you folks planning on getting this to a wider, more diverse audience? (Especially folks that are less-online, or in different cultures or social circles?)
Neat. I think I was mislead by the fact that I saw something like “80 values/100-something deliberations”, and assumed you had a much smaller sample size.
Here’s hoping for good results if/when you expand globally. The US is a bit of an echo chamber in some respects.
Finally got some time to sit down and try this thing. It's a good start.
I'm somewhat concerned about some of these value cards to be able to mislead people. I was presented with a choice between the values of "Truth-Seeking" vs. "Accuracy" or something of the like.
At a glance, it looked like the latter made more sense, since it described someone getting overwhelmed with information and then choosing to reduce the cognitive load by picking reliable sources based on reputation and trustworthiness. But looking more closely at the values as articulated, that's not something I'd support in the general case, since it's a shortcut for actually thinking about problems. Sometimes you have to take a shortcut for practical purposes, but it's not a complete substitute for thinking deeply about a problem.
If I had rushed through the tool, I probably would have endorsed the latter. Thinking about it more carefully, though, I endorsed the former instead, since I think it generalizes better.
How are you folks preventing the LLM from ending up with values exactly like this: ones that sound great at a glance, but end up being not what people want when they sit down and think about it more deeply?
Thanks for trying it out!
In the last step, users deliberate about whether a value is a more comprehensive version of another. This should address what you’re describing, and so far we’ve found that even paid participants give very thoughtful answers there.
The deliberation step is what I was describing.
Good to hear that people are giving thoughtful answers in practice. Paying them probably helps.
Your tool is currently broken. I can select a scenario but whenever I try to type a response it just says "Failed to update chat. Please try again."
https://imgur.com/iBdYAax
Thanks for letting us know, fixed now!
It would be cool to rate other values cards on some basic scale instead of just selecting the ones you like most.
-1 - I think this is actively bad
0 - I don't feel this is needed at all, but it's basically neutral
1 - I see the point and value for others but personally don't mind either way if it's included
2 - I think this is good but not completely necessary
3 - I think this is completely necessary
Looks like there are only a few dozen respondents so far. How are you folks planning on getting this to a wider, more diverse audience? (Especially folks that are less-online, or in different cultures or social circles?)
We’ve run this through 500+ people, including a representative sample of the US - which was the intention for step #1 in this experiment.
A results report is coming soon
Neat. I think I was mislead by the fact that I saw something like “80 values/100-something deliberations”, and assumed you had a much smaller sample size.
Here’s hoping for good results if/when you expand globally. The US is a bit of an echo chamber in some respects.
I had an error trying to sign up: https://imgur.com/a/LR8j2Uf
Should be fixed now, sorry about that!
The regenerate button blocks some text making it hard to go back and read it: https://imgur.com/a/KsmGRg9