[SERVER-1082] puppet file resource can't handle binary data Created: 2016/01/07
Updated:
2016/01/21
Status:
Project:
Component/s:
Affects
Version/s:
Fix Version/s:
Needs Information
Puppet Server
None
SERVER 2.2.1
Type:
Reporter:
Resolution:
Labels:
Remaining
Estimate:
Time Spent:
Original
Estimate:
Environment:
Bug
Daniel Urist
Unresolved
None
Not Specified
None
Priority:
Assignee:
Votes:
Normal
Daniel Urist
0
Not Specified
Not Specified
Debian Jessie, packages from puppetlabs repo:
puppetlabs-release-pc1 0.9.3.3.g91e66b9-1jessie
puppetserver 2.2.1-1puppetlabs1
stdlib 4.10.0
Issue Links:
Template:
Epic Link:
Scrum Team:
QA Contact:
Relates
relates to PUP-1031 Improve Puppet's handling of non-ASCI...
relates to PUP-3600 Create support for binary content on ...
Open
Open
customfield_10700 true
Unicode Encodings
Puppet Server
Erik Dasher
Description
I'm trying to distribute a binary key for corosync. I've generated it with corosync-keygen, then
base64-encoded it and stored it in hiera. I have the following resource in my manifest:
file
{ '/etc/corosync/authkey': content => base64('decode', $authkey), mode => '0400' }
This causes the puppet agent to time out, with no useful information in the puppetserver log.
I've verified that I can successfully encode/decode base64 strings with the base64() function
from stdlib, so I don't think that's the issue; I've tested against the current version of stdlib
(4.10.0).
The only other relevant issue I can find is this old one:
https://projects.puppetlabs.com/issues/5261
Comments
Comment by Christopher Price [ 2016/01/13 ]
Hi Daniel Urist, sorry for the delayed response on this.
Is this a pattern / approach that you have used in the past? Has it worked before, or is this the
first time you've tried it?
Also, might it be possible to do the decoding on the agent side somehow? Maybe via an exec
resource?
Comment by Daniel Urist [ 2016/01/13 ]
I am using an exec resource as a hacky workaround at the moment, but...
shouldn't the file resource handle binary data?
On Wed, Jan 13, 2016 at 11:44 AM, Christopher Price (JIRA) <
Comment by Christopher Price [ 2016/01/13 ]
It should be, though the support for that has been problematic at times in the past. I linked in
some other tickets describing future work around improving the situation, but those obviously
won't be of immediate use to you.
I know that there are users who do this in production, but it may be the case that they are doing
it via the file function, and/or via the source parameter on a file resource as opposed to the
content parameter. (I don't have a ton of experience with this, so, apologies for being a little
fuzzy on the details). I'm not aware of any reason why it shouldn't work with base64. It's
possible that it's a bug related to the specific bytes in question here?
Josh Cooper do you have any thoughts on this?
Daniel Urist was this behaving differently for you in previous versions of Puppet?
Comment by Daniel Urist [ 2016/01/14 ]
I believe (but am not entirely certain) that this was working until very
recently; I think it broke in the latest release of puppetserver.
On Wed, Jan 13, 2016 at 1:47 PM, Christopher Price (JIRA) <
Comment by Josh Cooper [ 2016/01/15 ]
The puppet file type can't handle all binary content unfortunately, see PUP-3600. If your binary
content appears to look like the start of a UTF-8 encoded string, then things will break. For
example, the euro symbol € in UTF-8 is the 3 byte sequence 0xE2 0x82 0xAC. If you have file
content that contains 0xE2 0x82 0x10, then puppet the agent will reject it when trying to
deserialize the catalog, since it's not a valid UTF-8 sequence:
Error: invalid byte sequence in UTF-8
If you really must transfer binary content, you are better off using the source parameter,
assuming you're not applying a template or other function.
Note puppet won't always fail with binary content, as it just depends on the binary content being
transferred.
That said, there could be a puppetserver issue here, as I would still expect the agent to get a
catalog, and fail to deserialize the catalog, as opposed to the agent hanging.
Comment by Eric Sorenson [ 2016/01/20 ]
Daniel Urist it would be very helpful if you could work up a small reproducer case - could you
please determine if this is actually true?
I believe (but am not entirely certain) that this was working until very recently; I
think it broke in the latest release of puppetserver.
As Josh Cooper notes, we don't have super awesome handling of utf-8 encoded data but the
agent timing out is not what I'd expect.
Comment by Daniel Urist [ 2016/01/21 ]
Well, I tried creating a test case, but now it seems to be working-- I have
no idea why.
On Wed, Jan 20, 2016 at 11:35 AM, Eric Sorenson (JIRA) <
Generated at Tue Feb 09 17:42:09 PST 2016 using JIRA 6.4.12#64027sha1:e3691cc1283c0f3cef6d65d3ea82d47743692b57.