With my previous (not correct) understanding, gnuplot can only handle data files separated by spaces. Therefore, whenever I wanted to visualize the csv files with gnuplot, I had to use awk or sed to remove the commas and then save the results into new files. It made my folders contain redundant data files which were just for using gnuplot. It's a little stupid, I always thought.
Today I encountered several csv files again, but this time I made up my mind to find the solution out. From the searching results with Google, I found there have been some people asking the same question but without easy solutions given. Then I found a document of old version gnuplot. It gives the right approach: to set datafile separator.
So, next time when you need to handle the csv files with gnuplot, remember to add the following line in the gnuplot script:
set datafile separator ","