These are called lower-thirds. What makes them “overlay” the background video is either the level of transparency or the transfer mode. So, if you have a solid rectangle on which the text resides, you can change the transparency from 100% to something less and it will begin to see-through the rectangle to show layers below it (presumably for you the video). Likewise, you can toggle on and off the transfer mode options (bottom left of the timeline window) or find them in the Layers menu. Each one interacts with the layers beneath it differently for many effects. All of these settings are also key-framable.
Transparency is easiest to access by clicking a layer and hitting “T” on the keyboard. It is also the easiest to work with as you start on this path.
Ramsey