Definitely stay in downtown Greenville.
I looked at Google maps to refresh my memory - if you want to be RIIIIIGHT on the river, book at either Embassy Suites or the Hampton Inn. The Embassy Suites is probably better, as it has a rooftop bar called Up on the Roof (there are several excellent rooftop bars within about 5 or 10 blocks of the river).
The Courtyard by Marriott is also nice, there is a very good restaurant down on the ground level called Rick Erwins, I went there a couple of times. I think the Westin is pretty nice too, it's a block north of the Courtyard.
You might also want to try Aloft, it's right by Tupelo Honey (loved that place). When I was up there a few years ago, there was a lot of new construction along Church Street, seems like there are some newer and/or renovated spots over there too.
Make a point to see Falls Park, and also the (minor league) baseball stadium a few blocks west, it is a miniature version of Fenway.