upload binlogs without helix token in AzDo

[pavelsavara]

publish-logs.yml should be used instead of generic log uploader, so that token would be scrubbed.

CheckHelixJobStatus is the offender I found on internal pipeline.

[premun]

If not whole publish-logs.yml then at least https://github.com/dotnet/arcade/blob/main/eng/common/post-build/redact-logs.ps1

[pavelsavara]

cc @LoopedBard3 @sebastienros

[LoopedBard3]

FYI, @DrewScoggins, @caaavik-msft as well.

[DrewScoggins]

Taking a look now.

[DrewScoggins]

So, these logs are coming from the Helix side. We place them in a folder that Helix controls and then uploads them for us to AzDO. We should just execute the redact-logs script as a post helix step, before they get uploaded.

I am a little surprised this isn't already covered by the Helix uploader. I see that our repo doesn't have a BinlogSecretsRedactionFile.txt if we added this file and put in the tokens we cared about would we get the redaction for free?

I misinterpreted what was happening here. I will take a look at where we are uploading these artifacts and use publish-logs.yml instead.

[DrewScoggins]

OK, after some more digging and a chat with some dnceng folks, it looks like this will be an Arcade fix. We never explicitly call the upload of artifacts job. It is just baked into the underlying job.yml that we use from Arcade. Given that, we will wait until we get a fix from Arcade.

[premun]

@DrewScoggins the pipeline that @pavelsavara sent me is not using job.yml and that is probably why the token got to the artifacts:
https://dev.azure.com/dnceng/internal/_build/results?buildId=2789939&view=logs&j=909ada5e-9b99-57ea-5db4-6842dbf85e5c&t=2366437b-a34b-5568-4fb5-e265d63baf65

[DrewScoggins]

I am pretty sure that we use the job.yml. If you look at run-performance-job.yml you can see that we use the job.yml as the jobTemplate. It is just that the default jop template just calls upload directly and doesn't call through the publish-logs.yml file. I have a change out that works for Windows, by calling redact-logs.ps1 directly. We are going to have to update the our Linux queues however, as they can't run a powershell script. I will also make the changes for the runtime runs.