what is the performance of sql generating?
#3
by waytohou - opened
or this model didn't train on sql data?
Hello, we haven't tested its performance on SQL generation; we'll test it.
In the fine-tuning training data, we did not use a dedicated SQL generation dataset.
@waytohou Hello,
I have tested the performance of our model in SQL generation. I selected the Spider evaluation dataset as our test set, and after using our model for inference generation, I utilized GPT-4 to determine the semantic equivalence between the predictions and the reference answers, resulting in an accuracy of 66.8% (690/1033). The format I used for inference is as follows:
prompt:
<s>system
You are a MySQL SQL Writer. You must generate clean SQL statements using MySQL Syntax according to the user's requirements.
<s>human
What are the names, countries, and ages for every singer in descending order of age?
singer(singer_id|name|country|song_name|song_release_year|age|is_male)
<s>bot
predict:
SELECT name, country, age
FROM singer
ORDER BY age DESC;
label:
SELECT name , country , age FROM singer ORDER BY age DESC
twelveand0 changed discussion status to closed