Skip to content

simplify using mimalloc#1454

Merged
dunglas merged 5 commits intophp:mainfrom
crazywhalecc:simple-mimalloc
Mar 25, 2025
Merged

simplify using mimalloc#1454
dunglas merged 5 commits intophp:mainfrom
crazywhalecc:simple-mimalloc

Conversation

@henderkes
Copy link
Copy Markdown
Contributor

work in progress

@henderkes henderkes marked this pull request as draft March 23, 2025 11:11
@henderkes
Copy link
Copy Markdown
Contributor Author

Well, I have no idea why it's trying to link mimalloc.o multiple times when it's only specified in CGO_LDFLAGS once.

@henderkes
Copy link
Copy Markdown
Contributor Author

@dunglas do you have an idea why the go compiler expands the command to include mimalloc.o multiple times? The CGO_LDFLAGS only has it once, when I compile my own C/C++ applications against the embed SAPI it works just fine, just the go compilation expands it to

/usr/bin/gcc -m64 -s -Wl,-z,relro -pie -Wl,--build-id=0x0b3f8aea96c13891237367c6562b32a3cb0d9665 -o $WORK/b001/exe/a.out -Wl,--export-dynamic-symbol=_cgo_panic -Wl,--export-dynamic-symbol=_cgo_topofstack -Wl,--export-dynamic-symbol=crosscall2 -Wl,--export-dynamic-symbol=go_apache_request_headers -Wl,--export-dynamic-symbol=go_frankenphp_after_script_execution -Wl,--export-dynamic-symbol=go_frankenphp_before_script_execution -Wl,--export-dynamic-symbol=go_frankenphp_finish_php_request -Wl,--export-dynamic-symbol=go_frankenphp_finish_worker_request -Wl,--export-dynamic-symbol=go_frankenphp_main_thread_is_ready -Wl,--export-dynamic-symbol=go_frankenphp_on_thread_shutdown -Wl,--export-dynamic-symbol=go_frankenphp_shutdown_main_thread -Wl,--export-dynamic-symbol=go_frankenphp_worker_handle_request_start -Wl,--export-dynamic-symbol=go_get_custom_php_ini -Wl,--export-dynamic-symbol=go_getenv -Wl,--export-dynamic-symbol=go_getfullenv -Wl,--export-dynamic-symbol=go_handle_file_watcher_event -Wl,--export-dynamic-symbol=go_is_context_done -Wl,--export-dynamic-symbol=go_log -Wl,--export-dynamic-symbol=go_putenv -Wl,--export-dynamic-symbol=go_read_cookies -Wl,--export-dynamic-symbol=go_read_post -Wl,--export-dynamic-symbol=go_register_variables -Wl,--export-dynamic-symbol=go_sapi_flush -Wl,--export-dynamic-symbol=go_ub_write -Wl,--export-dynamic-symbol=go_write_headers -Wl,--compress-debug-sections=zlib /tmp/go-link-290530713/go.o /tmp/go-link-290530713/000000.o /tmp/go-link-290530713/000001.o /tmp/go-link-290530713/000002.o /tmp/go-link-290530713/000003.o /tmp/go-link-290530713/000004.o /tmp/go-link-290530713/000005.o /tmp/go-link-290530713/000006.o /tmp/go-link-290530713/000007.o /tmp/go-link-290530713/000008.o /tmp/go-link-290530713/000009.o /tmp/go-link-290530713/000010.o /tmp/go-link-290530713/000011.o /tmp/go-link-290530713/000012.o /tmp/go-link-290530713/000013.o /tmp/go-link-290530713/000014.o /tmp/go-link-290530713/000015.o /tmp/go-link-290530713/000016.o /tmp/go-link-290530713/000017.o /tmp/go-link-290530713/000018.o /tmp/go-link-290530713/000019.o /tmp/go-link-290530713/000020.o /tmp/go-link-290530713/000021.o /tmp/go-link-290530713/000022.o /tmp/go-link-290530713/000023.o /tmp/go-link-290530713/000024.o /tmp/go-link-290530713/000025.o /tmp/go-link-290530713/000026.o /tmp/go-link-290530713/000027.o /tmp/go-link-290530713/000028.o /tmp/go-link-290530713/000029.o /tmp/go-link-290530713/000030.o /tmp/go-link-290530713/000031.o /tmp/go-link-290530713/000032.o /go/src/app/dist/static-php-cli/buildroot/lib/mimalloc.o -Wl,-O1 -pie /go/src/app/dist/static-php-cli/buildroot/lib/libbrotlicommon.a /go/src/app/dist/static-php-cli/buildroot/lib/libbrotlienc.a /go/src/app/dist/static-php-cli/buildroot/lib/libbrotlidec.a /go/src/app/dist/static-php-cli/buildroot/lib/libwatcher-c.a -L/go/src/app/dist/static-php-cli/buildroot/lib -lphp -lc -lyaml -ltidy -llz4 -lsqlite3 -lpq -lpgport -lpgcommon -lreadline -lonig -llber -lldap -lsodium -lMagick++-7.Q16HDRI -lMagickWand-7.Q16HDRI -lMagickCore-7.Q16HDRI -lzip -lheif -laom -lde265 -ltiff -lintl -lncurses -lgmp -lfreetype -ljpeg -lturbojpeg -lwebp -lwebpdecoder -lwebpdemux -lwebpmux -lsharpyuv -lavif -lpng16 -lcurl -lzstd -lnghttp2 -lxml2 -licui18n -licuio -licuuc -licudata -liconv -lcharset -lbrotlidec -lbrotlienc -lbrotlicommon -lssh2 -lssl -lcrypto -lz -lbz2 -lgomp -lstdc++ -lresolv -L/usr/local/lib -L/usr/lib -lphp -ldl -lm -lutil /go/src/app/dist/static-php-cli/buildroot/lib/mimalloc.o -Wl,-O1 -pie /go/src/app/dist/static-php-cli/buildroot/lib/libbrotlicommon.a /go/src/app/dist/static-php-cli/buildroot/lib/libbrotlienc.a /go/src/app/dist/static-php-cli/buildroot/lib/libbrotlidec.a /go/src/app/dist/static-php-cli/buildroot/lib/libwatcher-c.a -L/go/src/app/dist/static-php-cli/buildroot/lib -lphp -lc -lyaml -ltidy -llz4 -lsqlite3 -lpq -lpgport -lpgcommon -lreadline -lonig -llber -lldap -lsodium -lMagick++-7.Q16HDRI -lMagickWand-7.Q16HDRI -lMagickCore-7.Q16HDRI -lzip -lheif -laom -lde265 -ltiff -lintl -lncurses -lgmp -lfreetype -ljpeg -lturbojpeg -lwebp -lwebpdecoder -lwebpdemux -lwebpmux -lsharpyuv -lavif -lpng16 -lcurl -lzstd -lnghttp2 -lxml2 -licui18n -licuio -licuuc -licudata -liconv -lcharset -lbrotlidec -lbrotlienc -lbrotlicommon -lssh2 -lssl -lcrypto -lz -lbz2 -lgomp -lstdc++ -lbrotlicommon -lbrotlidec -lbrotlienc /go/src/app/dist/static-php-cli/buildroot/lib/mimalloc.o -Wl,-O1 -pie /go/src/app/dist/static-php-cli/buildroot/lib/libbrotlicommon.a /go/src/app/dist/static-php-cli/buildroot/lib/libbrotlienc.a /go/src/app/dist/static-php-cli/buildroot/lib/libbrotlidec.a /go/src/app/dist/static-php-cli/buildroot/lib/libwatcher-c.a -L/go/src/app/dist/static-php-cli/buildroot/lib -lphp -lc -lyaml -ltidy -llz4 -lsqlite3 -lpq -lpgport -lpgcommon -lreadline -lonig -llber -lldap -lsodium -lMagick++-7.Q16HDRI -lMagickWand-7.Q16HDRI -lMagickCore-7.Q16HDRI -lzip -lheif -laom -lde265 -ltiff -lintl -lncurses -lgmp -lfreetype -ljpeg -lturbojpeg -lwebp -lwebpdecoder -lwebpdemux -lwebpmux -lsharpyuv -lavif -lpng16 -lcurl -lzstd -lnghttp2 -lxml2 -licui18n -licuio -licuuc -licudata -liconv -lcharset -lbrotlidec -lbrotlienc -lbrotlicommon -lssh2 -lssl -lcrypto -lz -lbz2 -lgomp -lstdc++ -lpthread /go/src/app/dist/static-php-cli/buildroot/lib/mimalloc.o -Wl,-O1 -pie /go/src/app/dist/static-php-cli/buildroot/lib/libbrotlicommon.a /go/src/app/dist/static-php-cli/buildroot/lib/libbrotlienc.a /go/src/app/dist/static-php-cli/buildroot/lib/libbrotlidec.a /go/src/app/dist/static-php-cli/buildroot/lib/libwatcher-c.a -L/go/src/app/dist/static-php-cli/buildroot/lib -lphp -lc -lyaml -ltidy -llz4 -lsqlite3 -lpq -lpgport -lpgcommon -lreadline -lonig -llber -lldap -lsodium -lMagick++-7.Q16HDRI -lMagickWand-7.Q16HDRI -lMagickCore-7.Q16HDRI -lzip -lheif -laom -lde265 -ltiff -lintl -lncurses -lgmp -lfreetype -ljpeg -lturbojpeg -lwebp -lwebpdecoder -lwebpdemux -lwebpmux -lsharpyuv -lavif -lpng16 -lcurl -lzstd -lnghttp2 -lxml2 -licui18n -licuio -licuuc -licudata -liconv -lcharset -lbrotlidec -lbrotlienc -lbrotlicommon -lssh2 -lssl -lcrypto -lz -lbz2 -lgomp -lstdc++ /go/src/app/dist/static-php-cli/buildroot/lib/mimalloc.o -Wl,-O1 -pie /go/src/app/dist/static-php-cli/buildroot/lib/libbrotlicommon.a /go/src/app/dist/static-php-cli/buildroot/lib/libbrotlienc.a /go/src/app/dist/static-php-cli/buildroot/lib/libbrotlidec.a /go/src/app/dist/static-php-cli/buildroot/lib/libwatcher-c.a -L/go/src/app/dist/static-php-cli/buildroot/lib -lphp -lc -lyaml -ltidy -llz4 -lsqlite3 -lpq -lpgport -lpgcommon -lreadline -lonig -llber -lldap -lsodium -lMagick++-7.Q16HDRI -lMagickWand-7.Q16HDRI -lMagickCore-7.Q16HDRI -lzip -lheif -laom -lde265 -ltiff -lintl -lncurses -lgmp -lfreetype -ljpeg -lturbojpeg -lwebp -lwebpdecoder -lwebpdemux -lwebpmux -lsharpyuv -lavif -lpng16 -lcurl -lzstd -lnghttp2 -lxml2 -licui18n -licuio -licuuc -licudata -liconv -lcharset -lbrotlidec -lbrotlienc -lbrotlicommon -lssh2 -lssl -lcrypto -lz -lbz2 -lgomp -lstdc++ -lwatcher-c -lstdc++ -static-pie -Wl,-z,stack-size=0x80000

which contains it 5 times. I could use libmimalloc.a instead which would fix the issue, or use --allow-multiple-definition generally, but I'd rather fix the root issue of it being pulled in more than once.

… archives automatically, but it's slightly suboptimal. better would be to prevent cgo from duplicating it in the first place.
@henderkes
Copy link
Copy Markdown
Contributor Author

It works using the library instead of the object file because it still finds it before the first -lc in the command. It would only be necessary to use the .o file if -lc was specified earlier than libmimalloc.a due to linker precedence.

[root@localhost opt]# MIMALLOC_VERBOSE=1 ./frankenphp-linux-x86_64-mimalloc php-server
mimalloc: process init: 0x7FB87704F020
mimalloc: using 1 numa regions
mimalloc: v2.2.2, release (built on Mar 24 2025, 11:02:09)
mimalloc: option 'show_errors': 0
mimalloc: option 'show_stats': 0
mimalloc: option 'verbose': 1
mimalloc: option 'eager_commit': 1
mimalloc: option 'arena_eager_commit': 2
mimalloc: option 'purge_decommits': 1
mimalloc: option 'allow_large_os_pages': 2
mimalloc: option 'reserve_huge_os_pages': 0
mimalloc: option 'reserve_huge_os_pages_at': -1
mimalloc: option 'reserve_os_memory': 0 KiB
mimalloc: option 'deprecated_segment_cache': 0
mimalloc: option 'deprecated_page_reset': 0
mimalloc: option 'abandoned_page_purge': 0
mimalloc: option 'deprecated_segment_reset': 0
mimalloc: option 'eager_commit_delay': 1
mimalloc: option 'purge_delay': 10
mimalloc: option 'use_numa_nodes': 0
mimalloc: option 'disallow_os_alloc': 0
mimalloc: option 'os_tag': 100
mimalloc: option 'max_errors': 32
mimalloc: option 'max_warnings': 32
mimalloc: option 'max_segment_reclaim': 10
mimalloc: option 'destroy_on_exit': 0
mimalloc: option 'arena_reserve': 1048576 KiB
mimalloc: option 'arena_purge_mult': 10
mimalloc: option 'purge_extend_delay': 1
mimalloc: option 'abandoned_reclaim_on_free': 0
mimalloc: option 'disallow_arena_alloc': 0
mimalloc: option 'retry_on_oom': 400
mimalloc: option 'visit_abandoned': 0
mimalloc: option 'guarded_min': 0
mimalloc: option 'guarded_max': 1073741824
mimalloc: option 'guarded_precise': 0
mimalloc: option 'guarded_sample_rate': 0
mimalloc: option 'guarded_sample_seed': 0
mimalloc: option 'target_segments_per_thread': 0
mimalloc: option 'generic_collect': 10000
mimalloc: debug level : 0
mimalloc: secure level: 0
mimalloc: mem tracking: none
mimalloc: reserved 1048576 KiB memory

@henderkes henderkes marked this pull request as ready for review March 24, 2025 12:13
Comment thread build-static.sh Outdated
PHP_EXTENSION_LIBS="${PHP_EXTENSION_LIBS},brotli"
fi
# The mimalloc library must be built if MIMALLOC is true
if [ "${os}" = "linux" ] && [ -n "${MIMALLOC}" ]; then
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe could we drop the MIMALLOC env var entirely and use PHP_EXTENSION_LIBS directly?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would definitely be possible, but I didn't want to repeatedly list it as an extra lib in the deployments.

Using the mimalloc variable makes it clearer what actually happens and is backwards compatible to what people may already be using.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just realised that the check here should be spc_libc = musl though. Checking the OS makes no sense as both glibc linux and macos should not link mimalloc statically but preload the library.

@dunglas dunglas merged commit 8e6a183 into php:main Mar 25, 2025
41 of 42 checks passed
@dunglas
Copy link
Copy Markdown
Member

dunglas commented Mar 25, 2025

Thanks @DubbleClick

@henderkes henderkes deleted the simple-mimalloc branch March 25, 2025 15:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants