Yes, that's correct. All Bach wrote for the middle movement of this concerto in G major was a one-measure cadence iv6 to V in E minor (or a Phrygian cadence on B, if you like). Since the revival of Bach's music and the Brandenburg Concertos in particular, this movement has caused every orchestra and conductor massive headaches! What do you do? Just play the two-chord cadence as written? A few seconds of sustained chords in between two fast movements several minutes long each? That is obviously wrong, though it is what is written. What about a brief cadenza for the first violin? That is the option chosen by Tafelmusik and many others. Some have taken a slow movement from a different piece by Bach ending with the same cadence and simply inserted it here. The wildest interpretations have been those of Walter/Wendy Carlos in different realizations starting with the "Switched-On Bach" album performed on the Moog synthesizer and released in 1969 where the two chords were turned into a kind of fantasia with wild and wonderful sounds. Carlos did two other realizations of the Brandenburg No. 3 and each time created a new fantasia. Alas, I can't find any of these on YouTube, just the outer movements. To give you an idea, here is the first movement from the "Switched-On Bach" album:
I am mentioning this because, as odd as it might seem, I think that Carlos actually has the best idea of what to do with this movement. Given the kind of improvisatory skills that Bach and musicians he worked with had, isn't it really very obvious that what should be done here is take the opportunity to improvise something, which will be different in every concert? Isn't this what any gifted musician would have done well into the 19th century? These improvisational skills were, ironically, disappearing right around the time Bach's music was being rediscovered. The result is that musicians of our time look at these two chords and either literally just play what is there, or panic and stick in a movement from another piece! But it is perfectly obvious what is needed here. Some have, very conservatively, suggested that the harpsichord player or the first violin do a brief cadenza, which is rather too minimal. I suggest that the very obvious solution is for the violin, viola and cello to do a group improvisation for as long as it can be interestingly sustained. Two, three or more minutes. Have a listen to the cadenzas Mozart wrote for violin and viola together in his Sinfonia Concertante for a model. I'm sure that there are musicians that can do this. I once spent a whole afternoon improvising with a violist. If not, this is the perfect opportunity for musicians to practice their improvisatory skills. Sure, you can work out some general ideas beforehand, that's what rehearsals are for. But the idea of an improvised movement is so obviously what is called for here that I am shocked that no-one, apart from Carlos, has ever even tried to do it.
Here is what a lot of performances of that adagio sound like. A few seconds of a mini-cadenza for the first violin, going immediately into the last movement:
One final reason why I think that this is wildly insufficient is that the two outer movements are in G major, but these two chords of the adagio are on the dominant of the relative minor, E minor. There seems no point in setting up a new key if you aren't going to do anything in it, now is there?