AWS Access Policies & fog requirements

[i***@public.gmane.org]

Hi all,

Just getting familiar with fog. Really enjoy reading through the source so
far and am eager to get up to speed.

I'd like to attach an access policy to aws accounts and/or groups that
grants minimal access to the actions required for that account or group.
For example say I have an access policy attached to an account that allows
a limited set of actions to be taken on S3:

{
"Statement": [
{
"Action": [
"s3:GetBucketAcl",
"s3:GetBucketLocation",
"s3:GetBucketPolicy",
"s3:ListBucket",
"s3:GetObject",
"s3:GetObjectVersion"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::some.bucket/*"
]
}
]
}

Is this a use-case that fog takes into consideration? Or are there
constraints that require fog have unfettered access via the credentials
given it? Looking at
https://github.com/fog/fog/blob/master/lib/fog/aws/storage.rb it looks like
the latter is true but am new to fog so not sure if that's the right place
to determine what kind of access fog is requesting.

Basically I'd like my access policies to be a whitelist of actions &
resources allowed per account or group. Has anyone had success with this
approach?

Thanks!

- Jeremiah

[Frederick Cheung]

X-Received: by 10.180.39.114 with SMTP id o18mr315618wik.8.1358498692416;
Fri, 18 Jan 2013 00:44:52 -0800 (PST)
X-BeenThere: ruby-fog-/***@public.gmane.org
Received: by 10.180.101.165 with SMTP id fh5ls193246wib.13.gmail; Fri, 18 Jan
2013 00:44:51 -0800 (PST)
X-Received: by 10.180.19.194 with SMTP id h2mr1282025wie.0.1358498691068;
Fri, 18 Jan 2013 00:44:51 -0800 (PST)
X-Received: by 10.180.19.194 with SMTP id h2mr1282024wie.0.1358498691057;
Fri, 18 Jan 2013 00:44:51 -0800 (PST)
Received: from mail-wg0-f52.google.com (mail-wg0-f52.google.com [74.125.82.52])
by gmr-mx.google.com with ESMTPS id eu17si69229wid.0.2013.01.18.00.44.51
(version=TLSv1 cipher�DHE-RSA-RC4-SHA bits8/128);
Fri, 18 Jan 2013 00:44:51 -0800 (PST)
Received-SPF: pass (google.com: domain of frederick.cheung-***@public.gmane.org designates 74.125.82.52 as permitted sender) client-ipt.125.82.52;
Received: by mail-wg0-f52.google.com with SMTP id 12so2098112wgh.19
for <ruby-fog-/***@public.gmane.org>; Fri, 18 Jan 2013 00:44:51 -0800 (PST)
X-Received: by 10.194.123.105 with SMTP id lz9mr12689304wjb.43.1358498690983;
Fri, 18 Jan 2013 00:44:50 -0800 (PST)
Received: from [192.168.1.16] (fcheung.plus.com. [80.229.25.28])
by mx.google.com with ESMTPS id l5sm2688777wia.10.2013.01.18.00.44.49
(version=TLSv1 cipher�DHE-RSA-RC4-SHA bits8/128);
Fri, 18 Jan 2013 00:44:50 -0800 (PST)
In-Reply-To: <9843e0b1-7409-4246-a1ac-1dfdbea1c725-/***@public.gmane.org>
X-Mailer: iPhone Mail (10A551)
X-Original-Sender: frederick.cheung-***@public.gmane.org
X-Original-Authentication-Results: gmr-mx.google.com; spf=pass
(google.com: domain of frederick.cheung-***@public.gmane.org designates 74.125.82.52 as
permitted sender) smtp.mail=frederick.cheung-***@public.gmane.org; dkim=pass header.i=@gmail.com
Precedence: list
Mailing-list: list ruby-fog-/***@public.gmane.org; contact ruby-fog+owners-/***@public.gmane.org
List-ID: <ruby-fog.googlegroups.com>
X-Google-Group-Id: 920195093574
List-Post: <http://groups.google.com/group/ruby-fog/post?hl=en_US>, <mailto:ruby-fog-/***@public.gmane.org>
List-Help: <http://groups.google.com/support/?hl=en_US>, <mailto:ruby-fog+help-/***@public.gmane.org>
List-Archive: <http://groups.google.com/group/ruby-fog?hl=en_US>
Sender: ruby-fog-/***@public.gmane.org
List-Subscribe: <http://groups.google.com/group/ruby-fog/subscribe?hl=en_US>, <mailto:ruby-fog+subscribe-/***@public.gmane.org>
List-Unsubscribe: <http://groups.google.com/group/ruby-fog/subscribe?hl=en_US>,
<mailto:googlegroups-manage+920195093574+unsubscribe-/***@public.gmane.org>
Archived-At: <http://permalink.gmane.org/gmane.comp.lang.ruby.modules.fog/126>

Post by i***@public.gmane.org
{
"Statement": [
{
"Action": [
"s3:GetBucketAcl",
"s3:GetBucketLocation",
"s3:GetBucketPolicy",
"s3:ListBucket",
"s3:GetObject",
"s3:GetObjectVersion"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::some.bucket/*"
]
}
]
}
Is this a use-case that fog takes into consideration? Or are there constraints that require fog have unfettered access via the credentials given it? Looking at https://github.com/fog/fog/blob/master/lib/fog/aws/storage.rb it looks like the latter is true but am new to fog so not sure if that's the right place to determine what kind of access fog is requesting.

I use iam quite a lot with fog and haven't run into any problems I can recall

Fred

Post by i***@public.gmane.org
Basically I'd like my access policies to be a whitelist of actions & resources allowed per account or group. Has anyone had success with this approach?
Thanks!
- Jeremiah

[Jeremiah Heller]

X-Received: by 10.50.37.242 with SMTP id b18mr829787igk.6.1358527695593;
Fri, 18 Jan 2013 08:48:15 -0800 (PST)
X-BeenThere: ruby-fog-/***@public.gmane.org
Received: by 10.50.158.199 with SMTP id ww7ls698993igb.42.gmail; Fri, 18 Jan
2013 08:48:14 -0800 (PST)
X-Received: by 10.66.82.202 with SMTP id k10mr1354659pay.0.1358527694901;
Fri, 18 Jan 2013 08:48:14 -0800 (PST)
X-Received: by 10.66.82.202 with SMTP id k10mr1354658pay.0.1358527694888;
Fri, 18 Jan 2013 08:48:14 -0800 (PST)
Received: from mail-pa0-f51.google.com (mail-pa0-f51.google.com [209.85.220.51])
by gmr-mx.google.com with ESMTPS id vg4si998847pbc.2.2013.01.18.08.48.14
(version=TLSv1 cipher�DHE-RSA-RC4-SHA bits8/128);
Fri, 18 Jan 2013 08:48:14 -0800 (PST)
Received-SPF: pass (google.com: domain of ib.jeremiah-***@public.gmane.org designates 209.85.220.51 as permitted sender) client-ip 9.85.220.51;
Received: by mail-pa0-f51.google.com with SMTP id fb11so2216583pad.24
for <ruby-fog-/***@public.gmane.org>; Fri, 18 Jan 2013 08:48:14 -0800 (PST)
X-Received: by 10.66.72.198 with SMTP id f6mr25213574pav.42.1358527694822;
Fri, 18 Jan 2013 08:48:14 -0800 (PST)
Received: from [192.168.1.111] (76-14-200-79.or.wavecable.com. [76.14.200.79])
by mx.google.com with ESMTPS id jx6sm3377549pbc.13.2013.01.18.08.48.13
(version=TLSv1 cipher�DHE-RSA-RC4-SHA bits8/128);
Fri, 18 Jan 2013 08:48:14 -0800 (PST)
In-Reply-To: <D1DC35B4-FB48-44A4-8FB8-D8AA39616622-***@public.gmane.org>
X-Mailer: Apple Mail (2.1283)
X-Original-Sender: ib.jeremiah-***@public.gmane.org
X-Original-Authentication-Results: gmr-mx.google.com; spf=pass
(google.com: domain of ib.jeremiah-***@public.gmane.org designates 209.85.220.51 as
permitted sender) smtp.mail=ib.jeremiah-***@public.gmane.org; dkim=pass header.i=@gmail.com
Precedence: list
Mailing-list: list ruby-fog-/***@public.gmane.org; contact ruby-fog+owners-/***@public.gmane.org
List-ID: <ruby-fog.googlegroups.com>
X-Google-Group-Id: 920195093574
List-Post: <http://groups.google.com/group/ruby-fog/post?hl=en_US>, <mailto:ruby-fog-/***@public.gmane.org>
List-Help: <http://groups.google.com/support/?hl=en_US>, <mailto:ruby-fog+help-/***@public.gmane.org>
List-Archive: <http://groups.google.com/group/ruby-fog?hl=en_US>
Sender: ruby-fog-/***@public.gmane.org
List-Subscribe: <http://groups.google.com/group/ruby-fog/subscribe?hl=en_US>, <mailto:ruby-fog+subscribe-/***@public.gmane.org>
List-Unsubscribe: <http://groups.google.com/group/ruby-fog/subscribe?hl=en_US>,
<mailto:googlegroups-manage+920195093574+unsubscribe-/***@public.gmane.org>
Archived-At: <http://permalink.gmane.org/gmane.comp.lang.ruby.modules.fog/127>

Post by Frederick Cheung

Post by i***@public.gmane.org
{
"Statement": [
{
"Action": [
"s3:GetBucketAcl",
"s3:GetBucketLocation",
"s3:GetBucketPolicy",
"s3:ListBucket",
"s3:GetObject",
"s3:GetObjectVersion"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::some.bucket/*"
]
}
]
}
Is this a use-case that fog takes into consideration? Or are there constraints that require fog have unfettered access via the credentials given it? Looking at https://github.com/fog/fog/blob/master/lib/fog/aws/storage.rb it looks like the latter is true but am new to fog so not sure if that's the right place to determine what kind of access fog is requesting.

I use iam quite a lot with fog and haven't run into any problems I can recall

Thanks for the response. I'm coming to fog from RightAws. I have an access policy similar to my OP and attached to a group, which worked perfectly with RightAws. Attempting to use that same policy with fog consistently results in Excon::Errors::SocketError: Broken pipe (Errno::EPIPE). Updating the action list to ["s3:*"] fixes the Broken pipe error and fog behaves as expected. So I'm sure fog is requesting access that it doesn't really need.

I'll have another look later today and see if I can get a PR together to improve the error message and slim down permission requirements - does that sound reasonable?

Post by Frederick Cheung
Fred

Post by i***@public.gmane.org
Basically I'd like my access policies to be a whitelist of actions & resources allowed per account or group. Has anyone had success with this approach?
Thanks!
- Jeremiah

[Frederick Cheung]

X-Received: by 10.180.14.168 with SMTP id q8mr624897wic.3.1358527791910;
Fri, 18 Jan 2013 08:49:51 -0800 (PST)
X-BeenThere: ruby-fog-/***@public.gmane.org
Received: by 10.180.87.230 with SMTP id bb6ls390564wib.12.gmail; Fri, 18 Jan
2013 08:49:51 -0800 (PST)
X-Received: by 10.181.12.35 with SMTP id en3mr874154wid.2.1358527791374;
Fri, 18 Jan 2013 08:49:51 -0800 (PST)
X-Received: by 10.181.12.35 with SMTP id en3mr874153wid.2.1358527791362;
Fri, 18 Jan 2013 08:49:51 -0800 (PST)
Received: from mail-wg0-f49.google.com (mail-wg0-f49.google.com [74.125.82.49])
by gmr-mx.google.com with ESMTPS id cs2si312132wib.0.2013.01.18.08.49.51
(version=TLSv1 cipher�DHE-RSA-RC4-SHA bits8/128);
Fri, 18 Jan 2013 08:49:51 -0800 (PST)
Received-SPF: pass (google.com: domain of frederick.cheung-***@public.gmane.org designates 74.125.82.49 as permitted sender) client-ipt.125.82.49;
Received: by mail-wg0-f49.google.com with SMTP id 15so2272719wgd.4
for <ruby-fog-/***@public.gmane.org>; Fri, 18 Jan 2013 08:49:51 -0800 (PST)
X-Received: by 10.180.78.66 with SMTP id z2mr4730569wiw.23.1358527791270;
Fri, 18 Jan 2013 08:49:51 -0800 (PST)
Received: from [192.168.1.73] (fcheung.plus.com. [80.229.25.28])
by mx.google.com with ESMTPS id eo10sm4639974wib.9.2013.01.18.08.49.49
(version=TLSv1 cipher�DHE-RSA-RC4-SHA bits8/128);
Fri, 18 Jan 2013 08:49:50 -0800 (PST)
In-Reply-To: <1AC7E607-C47E-47F6-8680-E7EEC71C262C-***@public.gmane.org>
X-Mailer: Apple Mail (2.1499)
X-Original-Sender: frederick.cheung-***@public.gmane.org
X-Original-Authentication-Results: gmr-mx.google.com; spf=pass
(google.com: domain of frederick.cheung-***@public.gmane.org designates 74.125.82.49 as
permitted sender) smtp.mail=frederick.cheung-***@public.gmane.org; dkim=pass header.i=@gmail.com
Precedence: list
Mailing-list: list ruby-fog-/***@public.gmane.org; contact ruby-fog+owners-/***@public.gmane.org
List-ID: <ruby-fog.googlegroups.com>
X-Google-Group-Id: 920195093574
List-Post: <http://groups.google.com/group/ruby-fog/post?hl=en_US>, <mailto:ruby-fog-/***@public.gmane.org>
List-Help: <http://groups.google.com/support/?hl=en_US>, <mailto:ruby-fog+help-/***@public.gmane.org>
List-Archive: <http://groups.google.com/group/ruby-fog?hl=en_US>
Sender: ruby-fog-/***@public.gmane.org
List-Subscribe: <http://groups.google.com/group/ruby-fog/subscribe?hl=en_US>, <mailto:ruby-fog+subscribe-/***@public.gmane.org>
List-Unsubscribe: <http://groups.google.com/group/ruby-fog/subscribe?hl=en_US>,
<mailto:googlegroups-manage+920195093574+unsubscribe-/***@public.gmane.org>
Archived-At: <http://permalink.gmane.org/gmane.comp.lang.ruby.modules.fog/128>

Post by Jeremiah Heller

Post by Frederick Cheung

Post by i***@public.gmane.org
Is this a use-case that fog takes into consideration? Or are there constraints that require fog have unfettered access via the credentials given it? Looking at https://github.com/fog/fog/blob/master/lib/fog/aws/storage.rb it looks like the latter is true but am new to fog so not sure if that's the right place to determine what kind of access fog is requesting.

I use iam quite a lot with fog and haven't run into any problems I can recall

Thanks for the response. I'm coming to fog from RightAws. I have an access policy similar to my OP and attached to a group, which worked perfectly with RightAws. Attempting to use that same policy with fog consistently results in Excon::Errors::SocketError: Broken pipe (Errno::EPIPE). Updating the action list to ["s3:*"] fixes the Broken pipe error and fog behaves as expected. So I'm sure fog is requesting access that it doesn't really need.
I'll have another look later today and see if I can get a PR together to improve the error message and slim down permission requirements - does that sound reasonable?

Sure. Sounds weird though - when I've misconfigured policies the error messages have been explicit messages about authorisation being denied, not weird socket errors.

Fred

Post by Jeremiah Heller

Post by Frederick Cheung
Fred

Post by i***@public.gmane.org
Basically I'd like my access policies to be a whitelist of actions & resources allowed per account or group. Has anyone had success with this approach?
Thanks!
- Jeremiah

[Jeremiah Heller]

X-Received: by 10.50.7.244 with SMTP id m20mr1044230iga.14.1358536217793;
Fri, 18 Jan 2013 11:10:17 -0800 (PST)
X-BeenThere: ruby-fog-/***@public.gmane.org
Received: by 10.50.160.202 with SMTP id xm10ls841739igb.27.canary; Fri, 18 Jan
2013 11:10:17 -0800 (PST)
X-Received: by 10.66.82.37 with SMTP id f5mr1350077pay.27.1358536217255;
Fri, 18 Jan 2013 11:10:17 -0800 (PST)
X-Received: by 10.66.82.37 with SMTP id f5mr1350076pay.27.1358536217243;
Fri, 18 Jan 2013 11:10:17 -0800 (PST)
Received: from mail-da0-f52.google.com (mail-da0-f52.google.com [209.85.210.52])
by gmr-mx.google.com with ESMTPS id q1si1117327paz.0.2013.01.18.11.10.17
(version=TLSv1 cipher�DHE-RSA-RC4-SHA bits8/128);
Fri, 18 Jan 2013 11:10:17 -0800 (PST)
Received-SPF: pass (google.com: domain of ib.jeremiah-***@public.gmane.org designates 209.85.210.52 as permitted sender) client-ip 9.85.210.52;
Received: by mail-da0-f52.google.com with SMTP id f10so1747027dak.11
for <ruby-fog-/***@public.gmane.org>; Fri, 18 Jan 2013 11:10:17 -0800 (PST)
X-Received: by 10.68.189.66 with SMTP id gg2mr8091720pbc.111.1358536217122;
Fri, 18 Jan 2013 11:10:17 -0800 (PST)
Received: from [192.168.1.111] (76-14-200-79.or.wavecable.com. [76.14.200.79])
by mx.google.com with ESMTPS id w5sm3908541pax.28.2013.01.18.11.10.15
(version=TLSv1 cipher�DHE-RSA-RC4-SHA bits8/128);
Fri, 18 Jan 2013 11:10:16 -0800 (PST)
In-Reply-To: <E296B5F9-34D0-4F8C-A338-25E58C2F911A-***@public.gmane.org>
X-Mailer: Apple Mail (2.1283)
X-Original-Sender: ib.jeremiah-***@public.gmane.org
X-Original-Authentication-Results: gmr-mx.google.com; spf=pass
(google.com: domain of ib.jeremiah-***@public.gmane.org designates 209.85.210.52 as
permitted sender) smtp.mail=ib.jeremiah-***@public.gmane.org; dkim=pass header.i=@gmail.com
Precedence: list
Mailing-list: list ruby-fog-/***@public.gmane.org; contact ruby-fog+owners-/***@public.gmane.org
List-ID: <ruby-fog.googlegroups.com>
X-Google-Group-Id: 920195093574
List-Post: <http://groups.google.com/group/ruby-fog/post?hl=en_US>, <mailto:ruby-fog-/***@public.gmane.org>
List-Help: <http://groups.google.com/support/?hl=en_US>, <mailto:ruby-fog+help-/***@public.gmane.org>
List-Archive: <http://groups.google.com/group/ruby-fog?hl=en_US>
Sender: ruby-fog-/***@public.gmane.org
List-Subscribe: <http://groups.google.com/group/ruby-fog/subscribe?hl=en_US>, <mailto:ruby-fog+subscribe-/***@public.gmane.org>
List-Unsubscribe: <http://groups.google.com/group/ruby-fog/subscribe?hl=en_US>,
<mailto:googlegroups-manage+920195093574+unsubscribe-/***@public.gmane.org>
Archived-At: <http://permalink.gmane.org/gmane.comp.lang.ruby.modules.fog/129>

Post by Frederick Cheung

Post by Jeremiah Heller

Post by Frederick Cheung

Post by i***@public.gmane.org
Is this a use-case that fog takes into consideration? Or are there constraints that require fog have unfettered access via the credentials given it? Looking at https://github.com/fog/fog/blob/master/lib/fog/aws/storage.rb it looks like the latter is true but am new to fog so not sure if that's the right place to determine what kind of access fog is requesting.

I use iam quite a lot with fog and haven't run into any problems I can recall

Thanks for the response. I'm coming to fog from RightAws. I have an access policy similar to my OP and attached to a group, which worked perfectly with RightAws. Attempting to use that same policy with fog consistently results in Excon::Errors::SocketError: Broken pipe (Errno::EPIPE). Updating the action list to ["s3:*"] fixes the Broken pipe error and fog behaves as expected. So I'm sure fog is requesting access that it doesn't really need.
I'll have another look later today and see if I can get a PR together to improve the error message and slim down permission requirements - does that sound reasonable?

Sure. Sounds weird though - when I've misconfigured policies the error messages have been explicit messages about authorisation being denied, not weird socket errors.

Aha! Yes it is very weird behavior on Amazon's part.

The policy I had in place did not allow PutObjectAcl. Apparently Amazon will simply reset the connection in this case, rather than return a 401.

So there are 2 options to get around this: 1) set the x-amz-acl to ''; 2) make sure the access policy allows PutObjectAcl.

Unfortunately since Amazon resets the connection I don't see a way of improving this error message without polluting logs with warnings.

Is there a spot in the fog docs for this info?

Post by Frederick Cheung
Fred

Post by Jeremiah Heller

Post by Frederick Cheung
Fred

Post by i***@public.gmane.org
Basically I'd like my access policies to be a whitelist of actions & resources allowed per account or group. Has anyone had success with this approach?
Thanks!
- Jeremiah