True, I talk of dreams,
Which are the children of an idle brain,
Begot of nothing but vain fantasy,
Which is as thin of substance as the air
And more inconstant than the wind, who wooes
Even now the frozen bosom of the north,
And, being anger’d, puffs away from thence,
Turning his face to the dew-dropping south. 

 

I think that he is saying that he should just let go of the delusions of perfection that is holding to Rosaline. I think he is also saying that the delusions he  holding will slow him down in the real world.