#include #include #include #include #include #include #include #define DESCLEN 8 #define SCMIN 6 /* 1<<6 == 64 */ #define SCMAX 12 /* 1<<11 == 2048 */ const char *tagnames[] = { "TagInvalid", "TagKernelFs", "TagProcFs", "TagUserFs", "TagOpenPath", "TagMountPath", "TagMountUser", "TagMountRoot", "TagExecbuf", "TagVfsReq", "TagHandleset", "TagHandle", "TagProcess", "TagProcessHandle", "TagDevTime", "TagRootCache", "TagPageRefcount", }; static_assert(sizeof(tagnames) == sizeof(const char *) * TagLast); typedef struct Slab Slab; struct Slab { /* The slab is divided up into 1<next) { kprintf( "%08p %016llx %s %d\n", al, al->bitmap, full ? "full" : "part", 1ull<sizeclass ); assert(al->sizeclass == sizeclass); assert(al->bitmap & 1); assert(al->slot && *al->slot == al); assert(isfull(al->bitmap, sizeclass) == full); tally++; for (int i = 1; i * regsize < PAGE_SIZE; i++) { char *p = ((char*)al) + i * regsize; if (al->bitmap & (1ull<sizeclass && hdr->sizeclass < SCMAX); assert(hdr->bitmap & 1); return hdr; } void * kmalloc(size_t len, enum MallocTag tag) { Slab *al; void *ret; int sizeclass, regsize; uint64_t idx; assert(tag != TagInvalid && tag < TagLast); const char *desc = tagnames[tag] + 3; // very temporary assert(len <= KMALLOC_MAX); len += DESCLEN; sizeclass = getsizeclass(len); regsize = 1ull << sizeclass; if (freeslabs[sizeclass] == NULL) { al = page_alloc(1); al->sizeclass = sizeclass; al->bitmap = 1; /* mark header as allocated */ al->slot = &freeslabs[sizeclass]; al->next = NULL; freeslabs[sizeclass] = al; curslabs++; } al = freeslabs[sizeclass]; idx = findfreebit(al->bitmap); assert(1 <= idx && idx < 64); assert((al->bitmap & (1ull<bitmap |= 1ull<bitmap, sizeclass)) { *al->slot = al->next; al->next->slot = al->slot; al->next = fullslabs[sizeclass]; al->next->slot = &al->next; fullslabs[sizeclass] = al; al->slot = &fullslabs[sizeclass]; } return ret; } void kfree(void *ptr) { assert(ptr); Slab *al = getheader(ptr); int sizeclass = al->sizeclass; int regsize = 1ull << sizeclass; uint64_t off = ((uint64_t)ptr) & PAGE_MASK; uint64_t idx = off >> sizeclass; bool full = isfull(al->bitmap, sizeclass); assert(((char*)al) + idx * regsize == ptr); /* aligned? */ assert(1 <= idx); /* not the header? */ assert(al->bitmap & (1ull<bitmap &= ~(1ull<bitmap == 1) { /* we're empty */ *al->slot = al->next; al->next->slot = al->slot; page_free(al, 1); curslabs--; } else if (full) { /* we're no longer full */ *al->slot = al->next; al->next->slot = al->slot; al->next = freeslabs[sizeclass]; al->next->slot = &al->next; freeslabs[sizeclass] = al; al->slot = &freeslabs[sizeclass]; assert(fullslabs[sizeclass] != al); } }