Gambit's Stacked Deck doesn't end when The Silent King expires

I have a single 5* Gambit cover (Rajin' Cajun) and Black Bolt with two levels of The Silent King (also 1 level each of his other powers, if that somehow matters). When I use The Silent King on Gambit Stacked Deck causes Purple and Red powers not to work, as expected, but when the countdown for The Silent King ends Red and Purple powers continue not to work with the Prevented by Stacked Deck notification.