Scramming_Oscar comments on If you have nothing to hide, you have nothing to fear: This picture addresses surveillance (and the most intrusive form of surveillance imaginable) not information gathering.
So what is the danger of information gathering if it is not intrusive? The point is that it stifles dissent. People are put on no-fly lists (see for example Laura Poitras' story), get tax audits when they oppose the elites, are unable to work for the government because of their political opinion. Lets be honest, the government collects all this data not because all of it is interesting (95% of the people are boring twats, even to the government), but to filter out that 5% that is interesting.
That is why we need to be worried. It is dangerous not because it hurts everybody directly (well in a way it does, but not in the intrusive kind of way, more on a societal level), but because it directly undermines the meaning of free speech, free press, right to gather, right to be politically active. Privacy violated through information gathering is about power and control of political dissidents, not about some Mr Derpadorp having his breakfast and taking home a one night stand (unless, of course, Derpadorp is someone significant like MLK.