I always pictured a small mill as it was probably powered by Bill Skinner's "stream" and not some river so the Coldwater ON mill seems to fit the bill. Others that came to mind were the Elora Gorge Mill in Elora Ontario which now operates as an inn. Having stayed there overnight, I had my own 'Summertime Dreams'. Elora is pretty much a "Lunch pail town in a one horse way" situated on the banks of the Speed River if I recall.
The other mill is Roblin's mill located at Black Creek Pioneer Village in Toronto. Again, seems somewhat too large to fit the images conjured up in 'Summertime Dream'.
If not the Coldwater mill then something like McConnell's Mill in Pennsylvania. After all, I don't think that Gord specified it was a Canadian Mill.
Elora Mill Inn http://www.eloramill.com/?src=google...FQWuQAodvib-6Q
Black Creek Pioneer Village http://www.blackcreek.ca/