The gargled signal you’re getting maybe a looped signal (like this: signal from the deck’s input to your mac, from your mac’s output to the deck).
Solution for me: Sell the old Power Mac G4 w/ that DeckLink card and get another solution that works w/ my Power Mac G5. It seems no DeckLink card will work in my G5…