AWS Lambda の初期化フェーズ (INIT) は「10秒」に制限されている．例えば，AWS Lambda 関数の「ベストプラクティス」を意識してハンドラ外に実装した処理が長くなったり，AWS Lambda 関数 (Python) で機械学習系のライブラリなどを多く import しようとして遅くなり，結果的に INIT のタイムアウトが出てしまうことがあったりする．
INIT_REPORT Init Duration: 10020.95 ms Phase: init Status: timeout
AWS Lambda 関数 (Python) で import の最適化をするためには「そもそもどの import が遅いのか」を確認したく，import の前後に計測用のログを仕込むこともできるけど，Python 3.7 から使える -X importtime オプション（もしくは環境変数 PYTHONPROFILEIMPORTTIME）が便利で AWS Lambda 関数でも使えるので紹介したいと思う💡
今回は以下の「計4パターン」で試していく❗️
- AWS Lambda 関数 (ZIP) x
PYTHONPROFILEIMPORTTIME環境変数 - AWS Lambda 関数 (ZIP) x Layer x
-X importtimeオプション - AWS Lambda 関数 (Image) x
PYTHONPROFILEIMPORTTIME環境変数 - AWS Lambda 関数 (Image) x
-X importtimeオプション
サンプルコード
あくまでサンプルなので意味はないけど，今回は boto3 / json / numpy / requests を import するコードを準備した👌
👾 requirements.txt
boto3 numpy requests
👾 app.py
import boto3 import json import numpy as np import requests def lambda_handler(event, context): return { 'statusCode': 200, 'body': json.dumps({'message': 'hello world'}) }
1. AWS Lambda 関数 (ZIP) x PYTHONPROFILEIMPORTTIME 環境変数
👾 template.yaml
AWS SAM の template.yaml を以下のように書いた．
Environment.Variables で PYTHONPROFILEIMPORTTIME: 1 を設定している😃
AWSTemplateFormatVersion: 2010-09-09 Transform: AWS::Serverless-2016-10-31 Resources: Function: Type: AWS::Serverless::Function Properties: FunctionName: importtime-env CodeUri: src/ Handler: app.lambda_handler Runtime: python3.12 Architectures: - x86_64 Environment: Variables: PYTHONPROFILEIMPORTTIME: 1
AWS Lambda 関数 importtime-env をデプロイして実行すると以下のように import 時間がログに出力されていた❗️（ちなみに AWS CloudWatch Logs に出力されたログは表形式が崩れてしまっていて見にくかったため，以下のログは AWS SAM の sam local invoke コマンドの結果を載せている🙏）
import time: self [us] | cumulative | imported package import time: 1041 | 1041 | _io import time: 89 | 89 | marshal import time: 826 | 826 | posix import time: 2987 | 4941 | _frozen_importlib_external import time: 361 | 361 | time import time: 446 | 807 | zipimport import time: 33 | 33 | _codecs import time: 424 | 456 | codecs import time: 542 | 542 | encodings.aliases import time: 3553 | 4551 | encodings import time: 170 | 170 | encodings.utf_8 import time: 158 | 158 | _signal import time: 44 | 44 | _abc import time: 427 | 471 | abc import time: 857 | 1327 | io import time: 82 | 82 | _stat import time: 100 | 181 | stat import time: 1870 | 1870 | _collections_abc import time: 38 | 38 | genericpath import time: 82 | 120 | posixpath import time: 992 | 3162 | os import time: 76 | 76 | _sitebuiltins import time: 186 | 186 | sitecustomize import time: 53 | 53 | usercustomize import time: 1784 | 5259 | site import time: 391 | 391 | types import time: 65 | 65 | _operator import time: 492 | 557 | operator import time: 226 | 226 | itertools import time: 142 | 142 | keyword import time: 217 | 217 | reprlib import time: 105 | 105 | _collections import time: 739 | 1426 | collections import time: 87 | 87 | _functools import time: 1574 | 3087 | functools import time: 1797 | 5829 | enum import time: 70 | 70 | _sre import time: 288 | 288 | re._constants import time: 349 | 636 | re._parser import time: 215 | 215 | re._casefix import time: 373 | 1294 | re._compiler import time: 165 | 165 | copyreg import time: 1651 | 8938 | re import time: 906 | 906 | _json import time: 1298 | 2203 | json.scanner import time: 1010 | 12151 | json.decoder import time: 1093 | 1093 | json.encoder import time: 452 | 13694 | json import time: 142 | 142 | collections.abc import time: 152 | 152 | token import time: 56 | 56 | _tokenize import time: 1129 | 1337 | tokenize import time: 151 | 1487 | linecache import time: 890 | 890 | textwrap import time: 464 | 464 | contextlib import time: 671 | 3653 | traceback import time: 266 | 266 | warnings import time: 621 | 621 | _weakrefset import time: 479 | 1100 | weakref import time: 40 | 40 | _string import time: 569 | 609 | string import time: 580 | 580 | threading import time: 41 | 41 | atexit import time: 2290 | 8536 | logging import time: 174 | 174 | awslambdaric import time: 309 | 309 | importlib import time: 188 | 188 | awslambdaric.lambda_context import time: 97 | 97 | awslambdaric.lambda_runtime_exception import time: 1399 | 1399 | runtime_client import time: 289 | 289 | numbers import time: 979 | 1267 | _decimal import time: 134 | 1401 | decimal import time: 263 | 263 | math import time: 131 | 131 | __future__ import time: 211 | 211 | simplejson.errors import time: 123 | 123 | simplejson.raw_json import time: 192 | 192 | _struct import time: 111 | 303 | struct import time: 128 | 128 | simplejson.compat import time: 206 | 206 | simplejson._speedups import time: 156 | 362 | simplejson.scanner import time: 247 | 1040 | simplejson.decoder import time: 542 | 542 | simplejson.encoder import time: 368 | 2412 | simplejson import time: 232 | 4305 | awslambdaric.lambda_runtime_marshaller import time: 553 | 6353 | awslambdaric.lambda_runtime_client import time: 510 | 510 | awslambdaric.lambda_runtime_log_utils import time: 330 | 7689 | awslambdaric.bootstrap import time: 166 | 8028 | awslambdaric.__main__ import time: 684 | 684 | concurrent import time: 653 | 653 | concurrent.futures._base import time: 233 | 1569 | concurrent.futures import time: 150 | 150 | _heapq import time: 178 | 327 | heapq import time: 215 | 215 | _queue import time: 306 | 847 | queue import time: 298 | 1145 | concurrent.futures.thread import time: 225 | 225 | errno import time: 715 | 715 | _socket import time: 199 | 199 | select import time: 616 | 815 | selectors import time: 320 | 320 | array import time: 1888 | 3736 | socket import time: 2242 | 2242 | botocore import time: 1241 | 1241 | botocore.vendored import time: 2355 | 2355 | botocore.vendored.requests.packages.urllib3.exceptions import time: 1555 | 3910 | botocore.vendored.requests.packages.urllib3 import time: 1041 | 4950 | botocore.vendored.requests.packages import time: 18 | 4968 | botocore.vendored.requests.packages.urllib3 import time: 24 | 4991 | botocore.vendored.requests.packages.urllib3.exceptions import time: 1767 | 6758 | botocore.vendored.requests.exceptions import time: 1630 | 8387 | botocore.vendored.requests import time: 8803 | 20672 | botocore.exceptions import time: 2310 | 22981 | boto3.exceptions import time: 4605 | 31546 | boto3.compat import time: 208 | 208 | copy import time: 86 | 86 | _wmi import time: 491 | 576 | platform import time: 193 | 193 | _bisect import time: 132 | 325 | bisect import time: 189 | 189 | _random import time: 82 | 82 | _sha512 import time: 7551 | 7551 | _hashlib import time: 211 | 211 | _blake2 import time: 1203 | 8965 | hashlib import time: 932 | 10490 | random import time: 190 | 190 | _ast import time: 1253 | 1442 | ast import time: 160 | 160 | _opcode import time: 548 | 708 | opcode import time: 735 | 1443 | dis import time: 68 | 68 | importlib.machinery import time: 2190 | 5141 | inspect import time: 751 | 5892 | jmespath.compat import time: 1454 | 7346 | jmespath.exceptions import time: 1945 | 9291 | jmespath.lexer import time: 1121 | 1121 | jmespath.ast import time: 2542 | 2542 | jmespath.functions import time: 2458 | 5000 | jmespath.visitor import time: 5000 | 30900 | jmespath.parser import time: 1373 | 32273 | jmespath import time: 894 | 894 | botocore.docs.bcdoc import time: 570 | 570 | _datetime import time: 161 | 730 | datetime import time: 694 | 694 | http import time: 158 | 158 | email import time: 484 | 484 | email.errors import time: 218 | 218 | binascii import time: 291 | 291 | email.quoprimime import time: 344 | 344 | base64 import time: 192 | 535 | email.base64mime import time: 153 | 153 | quopri import time: 105 | 257 | email.encoders import time: 223 | 480 | email.charset import time: 695 | 2216 | email.header import time: 181 | 181 | urllib import time: 1539 | 1539 | ipaddress import time: 1116 | 2835 | urllib.parse import time: 87 | 87 | _locale import time: 700 | 786 | locale import time: 1121 | 1906 | calendar import time: 311 | 2217 | email._parseaddr import time: 503 | 5554 | email.utils import time: 321 | 8090 | email._policybase import time: 464 | 9038 | email.feedparser import time: 349 | 9544 | email.parser import time: 205 | 205 | email._encoded_words import time: 118 | 118 | email.iterators import time: 444 | 766 | email.message import time: 1176 | 1176 | _ssl import time: 2240 | 3415 | ssl import time: 916 | 15334 | http.client import time: 273 | 273 | shlex import time: 192 | 192 | importlib._abc import time: 137 | 328 | importlib.util import time: 6200 | 6527 | botocore.vendored.six import time: 2665 | 2665 | dateutil._version import time: 1312 | 3976 | dateutil import time: 4745 | 4745 | six import time: 45 | 45 | six.moves import time: 2056 | 2056 | dateutil.tz._common import time: 1089 | 1089 | dateutil.tz._factories import time: 29 | 29 | six.moves.winreg import time: 1994 | 2022 | dateutil.tz.win import time: 8267 | 18223 | dateutil.tz.tz import time: 912 | 23110 | dateutil.tz import time: 58 | 58 | _typing import time: 2975 | 3033 | typing import time: 5996 | 5996 | urllib3.exceptions import time: 1576 | 1576 | urllib3.util.timeout import time: 3763 | 5338 | urllib3.util.connection import time: 925 | 925 | urllib3.util.util import time: 100 | 100 | brotlicffi import time: 87 | 87 | brotli import time: 89 | 89 | zstandard import time: 2076 | 3275 | urllib3.util.request import time: 854 | 854 | urllib3.util.response import time: 2573 | 2573 | urllib3.util.retry import time: 252 | 252 | hmac import time: 8511 | 8511 | urllib3.util.url import time: 2046 | 2046 | urllib3.util.ssltransport import time: 2199 | 13006 | urllib3.util.ssl_ import time: 1111 | 1111 | urllib3.util.wait import time: 1007 | 27162 | urllib3.util import time: 30 | 27192 | urllib3.util.connection import time: 1851 | 29043 | urllib3._base_connection import time: 3180 | 3180 | urllib3._collections import time: 580 | 580 | urllib3._version import time: 78 | 78 | _winapi import time: 102 | 102 | winreg import time: 434 | 612 | mimetypes import time: 1646 | 2258 | urllib3.fields import time: 982 | 3240 | urllib3.filepost import time: 331 | 331 | zlib import time: 97 | 97 | brotlicffi import time: 85 | 85 | brotli import time: 84 | 84 | zstandard import time: 1295 | 1295 | urllib3.util.ssl_match_hostname import time: 3902 | 5197 | urllib3.connection import time: 5173 | 10965 | urllib3.response import time: 1379 | 15583 | urllib3._request_methods import time: 619 | 619 | urllib3.util.proxy import time: 3896 | 20097 | urllib3.connectionpool import time: 3176 | 3176 | urllib3.poolmanager import time: 98 | 98 | urllib3_secure_extra import time: 2025 | 67224 | urllib3 import time: 198 | 198 | xml import time: 219 | 416 | xml.etree import time: 368 | 368 | xml.etree.ElementPath import time: 543 | 543 | pyexpat import time: 333 | 875 | _elementtree import time: 746 | 1988 | xml.etree.ElementTree import time: 172 | 2575 | xml.etree.cElementTree import time: 88 | 88 | awscrt import time: 29 | 117 | awscrt.auth import time: 169 | 169 | _compression import time: 327 | 495 | gzip import time: 3166 | 119546 | botocore.compat import time: 824 | 824 | html.entities import time: 382 | 1206 | html import time: 406 | 406 | _markupbase import time: 1612 | 3223 | html.parser import time: 2229 | 5451 | botocore.docs.bcdoc.docstringparser import time: 2839 | 2839 | botocore.docs.bcdoc.style import time: 2818 | 131547 | botocore.docs.bcdoc.restdoc import time: 135 | 135 | fnmatch import time: 85 | 85 | _winapi import time: 79 | 79 | nt import time: 70 | 70 | nt import time: 82 | 82 | nt import time: 72 | 72 | nt import time: 73 | 73 | nt import time: 70 | 70 | nt import time: 131 | 659 | ntpath import time: 716 | 1509 | pathlib import time: 286 | 286 | _bz2 import time: 330 | 615 | bz2 import time: 378 | 378 | _lzma import time: 478 | 855 | lzma import time: 512 | 1982 | shutil import time: 315 | 2296 | tempfile import time: 175 | 175 | urllib.response import time: 208 | 382 | urllib.error import time: 1552 | 4229 | urllib.request import time: 1050 | 1050 | dateutil._common import time: 2968 | 4018 | dateutil.relativedelta import time: 7682 | 11699 | dateutil.parser._parser import time: 2059 | 2059 | dateutil.parser.isoparser import time: 1534 | 15291 | dateutil.parser import time: 3007 | 3007 | botocore.awsrequest import time: 920 | 920 | urllib3.contrib import time: 74 | 74 | OpenSSL import time: 25 | 99 | OpenSSL.SSL import time: 4066 | 5083 | urllib3.contrib.pyopenssl import time: 359 | 359 | importlib.resources.abc import time: 353 | 353 | importlib.resources._adapters import time: 477 | 1188 | importlib.resources._common import time: 223 | 223 | importlib.resources._legacy import time: 265 | 1675 | importlib.resources import time: 2328 | 4003 | certifi.core import time: 1107 | 5109 | certifi import time: 2877 | 13068 | botocore.httpsession import time: 15381 | 52482 | botocore.utils import time: 1084 | 53566 | botocore.docs.shape import time: 1522 | 1522 | botocore.docs.utils import time: 1627 | 56713 | botocore.docs.example import time: 1995 | 1995 | botocore.docs.params import time: 1450 | 3445 | botocore.docs.method import time: 1605 | 1605 | botocore.docs.sharedexample import time: 2610 | 64372 | botocore.docs.client import time: 1318 | 1318 | botocore.docs.paginator import time: 1093 | 1093 | botocore.docs.waiter import time: 3153 | 201481 | botocore.docs.service import time: 1636 | 203117 | botocore.docs import time: 1037 | 204154 | botocore.docs.docstring import time: 2504 | 238930 | botocore.waiter import time: 2959 | 2959 | botocore.eventstream import time: 4886 | 7844 | botocore.parsers import time: 2668 | 2668 | botocore.validate import time: 3865 | 6532 | botocore.serialize import time: 73 | 73 | _uuid import time: 529 | 602 | uuid import time: 979 | 979 | botocore.history import time: 2781 | 2781 | botocore.hooks import time: 1370 | 1370 | botocore.response import time: 2575 | 3944 | botocore.httpchecksum import time: 2545 | 10850 | botocore.endpoint import time: 1332 | 12182 | botocore.config import time: 5471 | 5471 | botocore.auth import time: 957 | 957 | botocore.crt import time: 3826 | 3826 | botocore.endpoint_provider import time: 3396 | 13649 | botocore.regions import time: 2541 | 2541 | botocore.signers import time: 37 | 37 | botocore.customizations import time: 26 | 63 | botocore.customizations.useragent import time: 2052 | 2115 | botocore.useragent import time: 2907 | 47767 | botocore.args import time: 1283 | 1283 | botocore.compress import time: 304 | 304 | termios import time: 200 | 503 | getpass import time: 484 | 484 | signal import time: 276 | 276 | fcntl import time: 89 | 89 | msvcrt import time: 169 | 169 | _posixsubprocess import time: 625 | 1641 | subprocess import time: 1175 | 1175 | configparser import time: 1404 | 2579 | botocore.configloader import time: 2315 | 2315 | botocore.tokens import time: 8087 | 15123 | botocore.credentials import time: 4083 | 4083 | botocore.model import time: 2047 | 6129 | botocore.discovery import time: 3074 | 3074 | botocore.paginate import time: 1002 | 1002 | botocore.retries import time: 1167 | 1167 | botocore.retries.bucket import time: 837 | 837 | botocore.retries.quota import time: 612 | 612 | botocore.retries.base import time: 882 | 1494 | botocore.retries.special import time: 2382 | 4712 | botocore.retries.standard import time: 873 | 873 | botocore.retries.throttling import time: 2707 | 9457 | botocore.retries.adaptive import time: 5685 | 328446 | botocore.client import time: 1741 | 1741 | botocore.retryhandler import time: 678 | 678 | botocore.translate import time: 5960 | 8378 | botocore.handlers import time: 2656 | 2656 | botocore.monitoring import time: 3620 | 3620 | botocore.configprovider import time: 1016 | 1016 | botocore.errorfactory import time: 1885 | 1885 | botocore.loaders import time: 5061 | 351634 | botocore.session import time: 939 | 939 | boto3.utils import time: 748 | 748 | boto3.resources import time: 738 | 738 | boto3.docs.client import time: 911 | 911 | boto3.docs.base import time: 878 | 878 | boto3.docs.method import time: 1310 | 1310 | boto3.docs.utils import time: 1322 | 4420 | boto3.docs.action import time: 812 | 812 | boto3.docs.attr import time: 1542 | 1542 | boto3.docs.collection import time: 1242 | 1242 | boto3.docs.subresource import time: 1202 | 1202 | boto3.docs.waiter import time: 2142 | 11359 | boto3.docs.resource import time: 4560 | 16656 | boto3.docs.service import time: 1104 | 17759 | boto3.docs import time: 967 | 967 | boto3.docs.docstring import time: 2357 | 2357 | boto3.resources.model import time: 1125 | 1125 | boto3.resources.params import time: 1329 | 1329 | boto3.resources.response import time: 1261 | 6071 | boto3.resources.action import time: 1101 | 1101 | boto3.resources.base import time: 1884 | 1884 | boto3.resources.collection import time: 4119 | 32645 | boto3.resources.factory import time: 2010 | 387435 | boto3.session import time: 3584 | 422564 | boto3 import time: 1792 | 1792 | numpy._utils._convertions import time: 1065 | 2856 | numpy._utils import time: 4987 | 7843 | numpy._globals import time: 1247 | 1247 | numpy.exceptions import time: 812 | 812 | numpy.version import time: 37 | 37 | numpy._distributor_init_local import time: 1031 | 1067 | numpy._distributor_init import time: 1391 | 1391 | numpy._utils._inspect import time: 4203 | 4203 | numpy.core._exceptions import time: 2220 | 2220 | numpy.dtypes import time: 47163 | 53585 | numpy.core._multiarray_umath import time: 2926 | 57902 | numpy.core.overrides import time: 7971 | 65872 | numpy.core.multiarray import time: 5783 | 5783 | numpy.core.umath import time: 2581 | 2581 | numpy.core._string_helpers import time: 213 | 213 | pickle5 import time: 577 | 577 | _compat_pickle import time: 977 | 977 | _pickle import time: 2721 | 4274 | pickle import time: 4231 | 8717 | numpy.compat.py3k import time: 3481 | 12197 | numpy.compat import time: 5065 | 5065 | numpy.core._dtype import time: 6900 | 24161 | numpy.core._type_aliases import time: 8037 | 34778 | numpy.core.numerictypes import time: 469 | 469 | _contextvars import time: 689 | 1157 | contextvars import time: 3892 | 5048 | numpy.core._ufunc_config import time: 7476 | 12524 | numpy.core._methods import time: 14225 | 26748 | numpy.core.fromnumeric import time: 7659 | 34407 | numpy.core.shape_base import time: 18204 | 18204 | numpy.core.arrayprint import time: 3078 | 3078 | numpy.core._asarray import time: 17565 | 73253 | numpy.core.numeric import time: 17894 | 17894 | numpy.core.defchararray import time: 10865 | 10865 | numpy.core.records import time: 8270 | 8270 | numpy.core.memmap import time: 8022 | 8022 | numpy.core.function_base import time: 10691 | 10691 | numpy.core._machar import time: 12212 | 12212 | numpy.core.getlimits import time: 18510 | 18510 | numpy.core.einsumfunc import time: 4183 | 4183 | numpy.core._multiarray_tests import time: 19182 | 23365 | numpy.core._add_newdocs import time: 2135 | 2135 | numpy.core._add_newdocs_scalars import time: 1353 | 1353 | numpy.core._dtype_ctypes import time: 512 | 512 | _ctypes import time: 290 | 290 | ctypes._endian import time: 2091 | 2892 | ctypes import time: 4432 | 7324 | numpy.core._internal import time: 1356 | 1356 | numpy._pytesttester import time: 8381 | 310056 | numpy.core import time: 34 | 310089 | numpy.core._multiarray_umath import time: 1585 | 311673 | numpy.__config__ import time: 6496 | 6496 | numpy.lib.mixins import time: 1105 | 1105 | numpy.lib.ufunclike import time: 2173 | 3278 | numpy.lib.type_check import time: 3751 | 7028 | numpy.lib.scimath import time: 1968 | 1968 | numpy.lib.stride_tricks import time: 2291 | 4259 | numpy.lib.twodim_base import time: 1709 | 1709 | numpy.linalg._umath_linalg import time: 3938 | 3938 | numpy._typing._nested_sequence import time: 668 | 668 | numpy._typing._nbit import time: 1909 | 1909 | numpy._typing._char_codes import time: 901 | 901 | numpy._typing._scalars import time: 579 | 579 | numpy._typing._shape import time: 2284 | 2284 | numpy._typing._dtype_like import time: 2896 | 2896 | numpy._typing._array_like import time: 1954 | 15125 | numpy._typing import time: 8540 | 29631 | numpy.linalg.linalg import time: 1296 | 30926 | numpy.linalg import time: 4079 | 35005 | numpy.matrixlib.defmatrix import time: 1447 | 36451 | numpy.matrixlib import time: 3048 | 3048 | numpy.lib.histograms import time: 13808 | 16856 | numpy.lib.function_base import time: 3018 | 56324 | numpy.lib.index_tricks import time: 3627 | 3627 | numpy.lib.nanfunctions import time: 2984 | 2984 | numpy.lib.shape_base import time: 4077 | 4077 | numpy.lib.polynomial import time: 4251 | 4251 | numpy.lib.utils import time: 2735 | 2735 | numpy.lib.arraysetops import time: 2799 | 2799 | numpy.lib.format import time: 2171 | 2171 | numpy.lib._datasource import time: 3395 | 3395 | numpy.lib._iotools import time: 7182 | 15546 | numpy.lib.npyio import time: 1428 | 1428 | numpy.lib.arrayterator import time: 2928 | 2928 | numpy.lib.arraypad import time: 1181 | 1181 | numpy.lib._version import time: 2177 | 110776 | numpy.lib import time: 943 | 943 | numpy.fft._pocketfft_internal import time: 3747 | 4690 | numpy.fft._pocketfft import time: 1083 | 1083 | numpy.fft.helper import time: 1119 | 6891 | numpy.fft import time: 2703 | 2703 | numpy.polynomial.polyutils import time: 3927 | 3927 | numpy.polynomial._polybase import time: 8857 | 15487 | numpy.polynomial.polynomial import time: 3503 | 3503 | numpy.polynomial.chebyshev import time: 2790 | 2790 | numpy.polynomial.legendre import time: 2856 | 2856 | numpy.polynomial.hermite import time: 2929 | 2929 | numpy.polynomial.hermite_e import time: 2790 | 2790 | numpy.polynomial.laguerre import time: 2001 | 32353 | numpy.polynomial import time: 78 | 78 | backports_abc import time: 4047 | 4124 | numpy.random._common import time: 172 | 172 | secrets import time: 2772 | 7067 | numpy.random.bit_generator import time: 1662 | 1662 | numpy.random._bounded_integers import time: 1472 | 1472 | numpy.random._mt19937 import time: 8317 | 18517 | numpy.random.mtrand import time: 1542 | 1542 | numpy.random._philox import time: 1664 | 1664 | numpy.random._pcg64 import time: 1384 | 1384 | numpy.random._sfc64 import time: 3873 | 3873 | numpy.random._generator import time: 4611 | 31589 | numpy.random._pickle import time: 1380 | 32968 | numpy.random import time: 2748 | 2748 | numpy.ctypeslib import time: 29009 | 29009 | numpy.ma.core import time: 6710 | 6710 | numpy.ma.extras import time: 1133 | 36852 | numpy.ma import time: 13048 | 558273 | numpy import time: 89 | 89 | chardet import time: 4947 | 4947 | charset_normalizer.constant import time: 1019 | 1019 | charset_normalizer.md__mypyc import time: 292 | 292 | unicodedata import time: 240 | 240 | _multibytecodec import time: 2500 | 3031 | charset_normalizer.utils import time: 2105 | 6154 | charset_normalizer.md import time: 3039 | 3039 | charset_normalizer.models import time: 2431 | 16570 | charset_normalizer.cd import time: 4711 | 21280 | charset_normalizer.api import time: 992 | 992 | charset_normalizer.legacy import time: 717 | 717 | charset_normalizer.version import time: 1189 | 24176 | charset_normalizer import time: 2043 | 2043 | http.cookiejar import time: 1044 | 1044 | http.cookies import time: 964 | 28314 | requests.compat import time: 6070 | 34383 | requests.exceptions import time: 110 | 110 | chardet import time: 70 | 70 | chardet import time: 2833 | 2833 | idna.package_data import time: 3689 | 3689 | idna.idnadata import time: 905 | 905 | idna.intranges import time: 3078 | 7671 | idna.core import time: 1230 | 11732 | idna import time: 1293 | 13094 | requests.packages import time: 96 | 96 | zipfile._path.glob import time: 401 | 497 | zipfile._path import time: 139 | 139 | zipfile.__main__ import time: 651 | 1285 | zipfile import time: 670 | 670 | requests.certs import time: 904 | 904 | requests.__version__ import time: 991 | 991 | requests._internal_utils import time: 2560 | 2560 | requests.cookies import time: 1027 | 1027 | requests.structures import time: 91 | 91 | importlib.resources._itertools import time: 287 | 378 | importlib.resources.readers import time: 144 | 522 | importlib.readers import time: 4215 | 12171 | requests.utils import time: 1928 | 1928 | requests.auth import time: 256 | 256 | stringprep import time: 277 | 533 | encodings.idna import time: 750 | 750 | requests.hooks import time: 1594 | 1594 | requests.status_codes import time: 3883 | 6759 | requests.models import time: 80 | 80 | socks import time: 1465 | 1545 | urllib3.contrib.socks import time: 2259 | 12489 | requests.adapters import time: 2922 | 15411 | requests.sessions import time: 940 | 16351 | requests.api import time: 2190 | 78298 | requests
2. AWS Lambda 関数 (ZIP) x Layer x -X importtime オプション
AWS Lambda では拡張機能 (Lambda extensions) を使ってランタイムに任意のオプションを追加することができる．以下のドキュメントにはラッパースクリプトを書いて Python ランタイムに -X importtime オプションを追加する例が載っている👀
👾 template.yaml
AWS SAM の template.yaml を以下のように書いた．
Python ランタイムに -X importtime オプションを追加するラッパースクリプトを含んだ AWS Lambda Layer を作る．そして AWS Lambda 関数の環境変数 AWS_LAMBDA_EXEC_WRAPPER に /opt/importtime_wrapper を設定する．
AWSTemplateFormatVersion: 2010-09-09 Transform: AWS::Serverless-2016-10-31 Resources: Function: Type: AWS::Serverless::Function Properties: FunctionName: importtime-wrapper CodeUri: src/ Handler: app.lambda_handler Runtime: python3.12 Architectures: - x86_64 Layers: - !Ref Layer Environment: Variables: AWS_LAMBDA_EXEC_WRAPPER: /opt/importtime_wrapper Layer: Type: AWS::Serverless::LayerVersion Properties: LayerName: importtime-wrapper-layer ContentUri: layer/src/ CompatibleRuntimes: - python3.12
👾 importtime_wrapper
importtime_wrapper はドキュメントの通りにしておく．そして，デプロイする前に忘れずに chmod +x layer/src/importtime_wrapper コマンドを実行してラッパースクリプトに実行権限を与えておく．
#!/bin/bash # the path to the interpreter and all of the originally intended arguments args=("$@") # the extra options to pass to the interpreter extra_args=("-X" "importtime") # insert the extra options args=("${args[@]:0:$#-1}" "${extra_args[@]}" "${args[@]: -1}") # start the runtime with the extra options exec "${args[@]}"
実行結果（割愛）
AWS Lambda 関数 importtime-wrapper をデプロイして実行すると同じく import 時間がログに出力されていた❗️
3. AWS Lambda 関数 (Image) x PYTHONPROFILEIMPORTTIME 環境変数
👾 template.yaml
AWS SAM の template.yaml を以下のように書いた．
Environment.Variables で PYTHONPROFILEIMPORTTIME: 1 を設定している😃
AWSTemplateFormatVersion: 2010-09-09 Transform: AWS::Serverless-2016-10-31 Resources: Function: Type: AWS::Serverless::Function Properties: FunctionName: importtime-env-image PackageType: Image Architectures: - x86_64 Environment: Variables: PYTHONPROFILEIMPORTTIME: 1 Metadata: Dockerfile: Dockerfile DockerContext: ./src DockerTag: python3.12-v1
👾 Dockerfile
FROM public.ecr.aws/lambda/python:3.12 COPY app.py requirements.txt ./ RUN python3.12 -m pip install -r requirements.txt -t . CMD ["app.lambda_handler"]
実行結果（割愛）
AWS Lambda 関数 importtime-env-image をデプロイして実行すると同じく import 時間がログに出力されていた❗️
4. AWS Lambda 関数 (Image) x -X importtime オプション
以下のドキュメントを参考に python:3.12 など一般的なコンテナイメージ (non-AWS base image) に AWS Lambda Runtime Interface Client (RIC) をインストールして実行するパターンを試す．Dockerfile のサンプルもドキュメントを参考にした．
👾 template.yaml
AWS SAM の template.yaml を以下のように書いた．
AWSTemplateFormatVersion: 2010-09-09 Transform: AWS::Serverless-2016-10-31 Resources: Function: Type: AWS::Serverless::Function Properties: FunctionName: importtime-option-image PackageType: Image Architectures: - x86_64 Metadata: Dockerfile: Dockerfile DockerContext: ./src DockerTag: python3.12-v1
👾 Dockerfile
ポイントは ENTRYPOINT に -X importtime オプションを追加しているところ📝
ARG FUNCTION_DIR="/function" FROM python:3.12 as build-image ARG FUNCTION_DIR RUN mkdir -p ${FUNCTION_DIR} COPY . ${FUNCTION_DIR} RUN pip install --target ${FUNCTION_DIR} -r ${FUNCTION_DIR}/requirements.txt FROM python:3.12-slim ARG FUNCTION_DIR WORKDIR ${FUNCTION_DIR} COPY --from=build-image ${FUNCTION_DIR} ${FUNCTION_DIR} ENTRYPOINT [ "/usr/local/bin/python", "-X", "importtime", "-m", "awslambdaric" ] CMD ["app.lambda_handler"]
👾 requirements.txt
requirements.txt に awslambdaric を追加した．
awslambdaric boto3 numpy requests
実行結果（割愛）
AWS Lambda 関数 importtime-option-image をデプロイして実行すると同じく import 時間がログに出力されていた❗️
まとめ
AWS Lambda 関数 (Python) で import の最適化をするときには Python の -X importtime オプション（もしくは環境変数 PYTHONPROFILEIMPORTTIME）が便利〜👏 という紹介記事でした ＼( 'ω')／
参考記事
以下の記事でも AWS Lambda 関数 (Python) に PYTHONPROFILEIMPORTTIME 環境変数を設定して INIT を最適化する方法が紹介されているので合わせて読んでみると良いかも📝
