Sure you can. Your best bet would be to track the movement of the sign in your video and apply the tracking to a null layer. Then create a new adjustment layer and pickwip the position of the adjustment layer to the null. Then apply a blur effect to the adjustment layer and mask out the effect using the mask tools so that only the sign is blurred. If you have done it right, the blur effect should track the sign in your footage.
As for people walking in front, you have a couple of options. One would be to key frame the opacity of the adjustment layer. Two, and probably the best, would be to key frame the mask of the adjustment layer. Third, you could duplicate your footage and rotoscope the person walking in front and place the rotoed footage above the adjustment layer.
Good luck