【AWS】API Gateway创建Rest API--从S3下载文件

一、背景

在不给AK,SK的前提下，用户查看s3上文件（从s3下载文件）

二、创建API

1、打开API Gateway，点击创建API，选择REST API

REST API和HTTP API区别：（来自AWS官网）

REST API 和 HTTP API 都是 RESTful API 产品。REST API 支持的功能比 HTTP API 多，而 HTTP API 在设计时功能就极少，因此能够以更低的价格提供。如果您需要如 API 密钥、每客户端节流、请求验证、AWS WAF 集成或私有 API 端点等功能，请选择 REST API。如果您不需要 REST API 中包含的功能，请选择 HTTP API。

2、 设置API名称，选择终端节点类型

终端节点类型：（来自AWS官网）

    区域性（REGIONAL）：适用于同一区域中的客户端。当在 EC2 实例上运行的客户端调用同一区域中的 API，或 API 用于为具有高需求的少数客户端提供服务时，区域 API 可以降低连接开销。

    还有边缘优化（EDGE）：最适合地理位置分散的客户端。API 请求将路由到最近的 CloudFront 接入点 (POP)。这是 API Gateway REST API 的默认终端节点类型。

    私有（PRIVATE）：是一个只能使用接口 VPC 终端节点从 Amazon Virtual Private Cloud (VPC) 访问的 API 终端节点，该接口是您在 VPC 中创建的终端节点网络接口 (ENI)

三、配置API

1、进入刚创建好的API，点击资源页，创建资源及方法

1.1创建资源， / 代表根目录，右击选择创建资源

1.2创建方法，上传文件到s3，所以选择GET方法

1.3点击刚创建的方法，进入集成请求

1.3配置：

集成类型：AWS 服务

AWS区域：（选择相应的区域）

AWS服务：S3

AWS子域：（不用填）

HTTP方法：GET

操作类型：路径覆盖

路径覆盖（可选）：{bucket}/{object}

执行角色：（填写有执行API角色的ARN）

路径覆盖也可以把路径某部分hard code

在下方URL路径参数中填写路径参数映射关系

2、配置设置

翻到最下面，修改二进制媒体类型为 /

Content-Encoding可以根据需要修改

默认上传文件大小不超过10M

三、添加授权方

1、新建Lambda函数，来验证授权方，运行时选择 Node.js 16.x

代码如下：当header中account和password匹配上，则allow，否则deny

exports.handler = function(event, context, callback) {        
    console.log('Received event:', JSON.stringify(event, null, 2));

    // A simple request-based authorizer example to demonstrate how to use request 
    // parameters to allow or deny a request. In this example, a request is  
    // authorized if the client-supplied headerauth1 header, QueryString1
    // query parameter, and stage variable of StageVar1 all match
    // specified values of 'headerValue1', 'queryValue1', and 'stageValue1',
    // respectively.

    // Retrieve request parameters from the Lambda function input:
    var headers = event.headers;
    var queryStringParameters = event.queryStringParameters;
    var pathParameters = event.pathParameters;
    var stageVariables = event.stageVariables;
        
    // Parse the input for the parameter values
    var tmp = event.methodArn.split(':');
    var apiGatewayArnTmp = tmp[5].split('/');
    var awsAccountId = tmp[4];
    var region = tmp[3];
    var restApiId = apiGatewayArnTmp[0];
    var stage = apiGatewayArnTmp[1];
    var method = apiGatewayArnTmp[2];
    var resource = '/'; // root resource
    if (apiGatewayArnTmp[3]) {
        resource += apiGatewayArnTmp[3];
    }
        
    // Perform authorization to return the Allow policy for correct parameters and 
    // the 'Unauthorized' error, otherwise.
    var authResponse = {};
    var condition = {};
    condition.IpAddress = {};
     
    if (headers.account === ""
        && headers.password === "") {
        callback(null, generateAllow('me', event.methodArn));
    }else {
        callback("Unauthorized");
    }
}
     
// Help function to generate an IAM policy
var generatePolicy = function(principalId, effect, resource) {
    // Required output:
    var authResponse = {};
    authResponse.principalId = principalId;
    if (effect && resource) {
        var policyDocument = {};
        policyDocument.Version = '2012-10-17'; // default version
        policyDocument.Statement = [];
        var statementOne = {};
        statementOne.Action = 'execute-api:Invoke'; // default action
        statementOne.Effect = effect;
        statementOne.Resource = resource;
        policyDocument.Statement[0] = statementOne;
        authResponse.policyDocument = policyDocument;
    }
    // Optional output with custom properties of the String, Number or Boolean type.
    authResponse.context = {
        "account": '',
        "password": '',
        "booleanKey": true
    };
    return authResponse;
}
     
var generateAllow = function(principalId, resource) {
    return generatePolicy(principalId, 'Allow', resource);
}
     
var generateDeny = function(principalId, resource) {
    return generatePolicy(principalId, 'Deny', resource);
}

2、创建授权方

授权方名称

类型：选择Lambda

Lambda函数：填写刚创建好的Lambda函数名称

Lambda调用角色：填写调用Lambda函数的角色

Lambda事件负载：选择请求

身份来源：选择标头，添加account和password

授权缓存：取消启用

三、配置授权方

选择 添加授权方的路径资源方法中的方法请求

授权选择配置好的授权方名称

请求验证程序:无

需要API密钥:否

HTTP请求标头:将account和password配置进来

四、部署API

API配置完成后，右击根目录，部署API， 选择部署阶段，点击部署

注意：每次对API进行更改后要重新部署一下

五、测试API

测试通过两种方式：①Postman ②python代码

获取URL链接

1、Postman

进入Postman，添加PUT请求，复制URL链接，在其后添加要下载文件的S3的路径，点击send，即可在下方看到请求结果

2、python代码

import json
import requests

def call_get_api(_url,_headers):
    res = requests.get(url=_url, headers=_headers)
    return res

def download_s3(bucket,key,local_file):
    # api gateway call url
    url_ip = ""
    # generate the url
    url = url_ip + bucket + key

    # headers
    headers = {"account": "", "password": ""}

    # call the api2s3 method
    res = call_get_api(url, headers)
    res.encoding = 'utf-8'
    data = res.text
    if res.status_code == 200:
        print(res.status_code)
        print(data)
        with open(local_file, 'wb') as f:
            # str通过encode()方法可以转换为bytes
            f.write(data.encode())
    else:
        print(res)

if __name__ == '__main__':
    # s3 file
    bucket = ''
    key = ''

    # local file name
    local_file = ''
    download_s3(bucket, key, local_file)

六、通过CloudFormation新建API

yaml文件代码如下

AWSTemplateFormatVersion: '2010-09-09'
Description : Template to provision ETL Workflow for api gateway 
Parameters:
  Region:
    Description: 'Specify the Region for resource.'
    Type: String
    Default: ase1
  Iteration:
    Type: String
    Description: 'Specify the Iteration for Lambda.'
    Default: '001'
  S3Iteration:
    Type: String
    Description: 'Specify the Iteration for S3'
    Default: '001'
  IAMIteration:
    Type: String
    Description: 'Specify the Iteration for IAM roles.'
    Default: '001'

Resources: 
  ApigatewayRestAPI:
    Type: AWS::ApiGateway::RestApi
    Properties:
      Name: api-downloads3-${Iteration}
      BinaryMediaTypes:
        - "*/*"
      Description: create api to download file from s3
      Mode: overwrite
      EndpointConfiguration:
        Types:
          - REGIONAL
 
  ApigatewayAuthorizer:
    Type: AWS::ApiGateway::Authorizer
    Properties:
      AuthorizerCredentials: "arn:aws:iam::${AWS::AccountId}:role/iamr-replication-${IAMIteration}"
      AuthorizerResultTtlInSeconds : 0
      AuthorizerUri: "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/arn:aws:lambda:${AWS::Region}:${AWS::AccountId}:function:lamb-apigw-authorizer-${S3Iteration}/invocations"
      Type : REQUEST
      AuthType: custom
      RestApiId: 
        !Ref  ApigatewayRestAPI
      Name: auth-request
      IdentitySource : method.request.header.account,method.request.header.password

  ApigatewayResourceFolder:
    Type: AWS::ApiGateway::Resource
    Properties:
      RestApiId: 
        !Ref ApigatewayRestAPI
      PathPart: "{folder}"
      ParentId: !GetAtt 
        - ApigatewayRestAPI
        - RootResourceId

  ApigatewayMethodFolder:
    Type: AWS::ApiGateway::Method
    Properties:
      AuthorizerId:
        !Ref ApigatewayAuthorizer
      AuthorizationType: custom
      RequestParameters: {
        "method.request.path.folder": true,
        "method.request.header.account": true,
        "method.request.header.password": true
      }
      HttpMethod: GET
      MethodResponses:
        - StatusCode: 200
          ResponseModels: 
            application/json: Empty
      RestApiId:
        !Ref ApigatewayRestAPI
      ResourceId: !GetAtt 
        - ApigatewayResourceFolder
        - ResourceId 
      Integration:
        Type: AWS
        Credentials: "arn:aws:iam::${AWS::AccountId}:role/iamr-replication-${IAMIteration}"
        IntegrationHttpMethod: GET
        IntegrationResponses:
          - StatusCode: 200
        PassthroughBehavior: when_no_match
        Uri: "arn:aws:apigateway:${AWS::Region}:s3:path/{folder}"
        RequestParameters: {
            "integration.request.path.folder" : "method.request.path.folder"
          }

  ApigatewayResourceTablename:
    Type: AWS::ApiGateway::Resource
    Properties:
      RestApiId: 
        !Ref ApigatewayRestAPI
      PathPart: "{tablename}"
      ParentId:
        !Ref ApigatewayResourceFolder

  ApigatewayMethodTablename:
    Type: AWS::ApiGateway::Method
    Properties:
      AuthorizerId:
        !Ref ApigatewayAuthorizer
      AuthorizationType: custom
      RequestParameters: {
        "method.request.path.folder": true,
        "method.request.path.tablename": true,
        "method.request.header.account": true,
        "method.request.header.password": true
      }
      HttpMethod: GET
      MethodResponses:
        - StatusCode: 200
          ResponseModels: 
            application/json: Empty
      RestApiId:
        !Ref ApigatewayRestAPI
      ResourceId: !GetAtt 
        - ApigatewayResourceTablename
        - ResourceId 
      Integration:
        Type: AWS
        Credentials: "arn:aws:iam::${AWS::AccountId}:role/iamr-replication-${IAMIteration}"
        IntegrationHttpMethod: GET
        IntegrationResponses:
          - StatusCode: 200
        PassthroughBehavior: when_no_match
        Uri: "arn:aws:apigateway:${AWS::Region}:s3:path/{folder}/{tablename}"
        RequestParameters: {
            "integration.request.path.folder" : "method.request.path.folder",
            "integration.request.path.tablename" : "method.request.path.tablename"
          }

  ApigatewayResourcePartition:
    Type: AWS::ApiGateway::Resource
    Properties:
      RestApiId:
        !Ref ApigatewayRestAPI
      PathPart: "{partition}"
      ParentId:
        !Ref ApigatewayResourceTablename

  ApigatewayMethodPartition:
    Type: AWS::ApiGateway::Method
    Properties:
      AuthorizerId:
        !Ref ApigatewayAuthorizer
      AuthorizationType: custom
      RequestParameters: {
        "method.request.path.folder": true,
        "method.request.path.tablename": true,
        "method.request.path.partition": true,
        "method.request.header.account": true,
        "method.request.header.password": true
      }
      HttpMethod: GET
      MethodResponses:
        - StatusCode: 200
          ResponseModels: 
            application/json: Empty
      RestApiId:
        !Ref ApigatewayRestAPI
      ResourceId: !GetAtt 
        - ApigatewayResourcePartition
        - ResourceId 
      Integration:
        Type: AWS
        Credentials: "arn:aws:iam::${AWS::AccountId}:role/iamr-replication-${IAMIteration}"
        IntegrationHttpMethod: GET
        IntegrationResponses:
          - StatusCode: 200
        PassthroughBehavior: when_no_match
        Uri: "arn:aws:apigateway:${AWS::Region}:s3:path/{folder}/{tablename}/{partition}"
        RequestParameters: {
            "integration.request.path.partition" : "method.request.path.partition",
            "integration.request.path.folder" : "method.request.path.folder",
            "integration.request.path.tablename" : "method.request.path.tablename"
          }

  ApigatewayResourceFilename:
    Type: AWS::ApiGateway::Resource
    Properties:
      RestApiId: 
        !Ref ApigatewayRestAPI
      PathPart: "{filename}"
      ParentId:
        !Ref ApigatewayResourcePartition

  ApigatewayMethodFilename:
    Type: AWS::ApiGateway::Method
    Properties:
      AuthorizerId:
        !Ref ApigatewayAuthorizer
      AuthorizationType: custom
      RequestParameters: {
        "method.request.path.folder": true,
        "method.request.path.tablename": true,
        "method.request.path.partition": true,
        "method.request.path.filename": true,
        "method.request.header.account": true,
        "method.request.header.password": true
      }
      HttpMethod: GET
      MethodResponses:
        - StatusCode: 200
          ResponseModels: 
            application/json: Empty
      RestApiId:
        !Ref ApigatewayRestAPI
      ResourceId: !GetAtt 
        - ApigatewayResourceFilename
        - ResourceId 
      Integration:
        Type: AWS
        Credentials: "arn:aws:iam::${AWS::AccountId}:role/iamr-replication-${IAMIteration}"
        IntegrationHttpMethod: GET
        IntegrationResponses:
          - StatusCode: 200
        PassthroughBehavior: when_no_match
        Uri: "arn:aws:apigateway:${AWS::Region}:s3:path/{folder}/{tablename}/{partition}/{filename}"
        RequestParameters: {
            "integration.request.path.partition" : "method.request.path.partition",
            "integration.request.path.filename" : "method.request.path.filename",
            "integration.request.path.folder" : "method.request.path.folder",
            "integration.request.path.tablename" : "method.request.path.tablename"
          }
          
  ApigatewayDeploymentv1:
    DependsOn: ApigatewayMethodFilename
    Type: AWS::ApiGateway::Deployment
    Properties: 
      RestApiId:
        !Ref ApigatewayRestAPI
      StageName : v1

  PermissionToInvokeLambda: 
    Type: AWS::Lambda::Permission
    Properties: 
      FunctionName: lamb-apigw-authorizer-${Iteration}
      Action: "lambda:InvokeFunction"
      Principal: "apigateway.amazonaws.com"
      SourceArn: !Sub
        - "arn:aws:execute-api:${AWS::Region}:${AWS::AccountId}:${APiId}/authorizers/${AuthorizerId}"
        - APiId: 
            !Ref ApigatewayRestAPI
          AuthorizerId: 
            !Ref ApigatewayAuthorizer

Outputs:
  RootResourceId:
    Value: !GetAtt ApigatewayRestAPI.RootResourceId
  AuthorizerId:
    Value: !GetAtt ApigatewayAuthorizer.AuthorizerId