From: Trond Myklebust --=-88DAsEQoFfsX4cDYh9m3 Content-Type: text/plain Content-Transfer-Encoding: 7bit RPC: add fair queueing to the RPC scheduler. If a wait queue is defined as a "priority queue" then requests are dequeued in blocks of 16 in order to work well with write gathering + readahead on the server. There are 3 levels of priority. The high priority tasks get scheduled 16 times for each time the default level gets scheduled. The lowest level gets scheduled once every 4 times the normal level gets scheduled. Original patch contributed by Shantanu Goel. Cheers, Trond --=-88DAsEQoFfsX4cDYh9m3 Content-Disposition: attachment; filename=linux-2.6.4-06-rpc_throttle.dif Content-Transfer-Encoding: base64 Content-Type: text/plain; name=linux-2.6.4-06-rpc_throttle.dif; charset=ISO-8859-1 IGZzL25mcy9uZnM0c3RhdGUuYyAgICAgICAgICAgICB8ICAgIDINCiBmcy9uZnMvcmVhZC5jICAg ICAgICAgICAgICAgICAgfCAgICAxDQogZnMvbmZzL3dyaXRlLmMgICAgICAgICAgICAgICAgIHwg ICA1MCArKysrKysrLS0tDQogaW5jbHVkZS9saW51eC9uZnNfZnMuaCAgICAgICAgIHwgICAxMiAr LQ0KIGluY2x1ZGUvbGludXgvc3VucnBjL3NjaGVkLmggICB8ICAgNjYgKysrKysrKysrKy0tLQ0K IG5ldC9zdW5ycGMvYXV0aF9nc3MvYXV0aF9nc3MuYyB8ICAgIDINCiBuZXQvc3VucnBjL2NsbnQu YyAgICAgICAgICAgICAgfCAgICAyDQogbmV0L3N1bnJwYy9zY2hlZC5jICAgICAgICAgICAgIHwg IDIwNCArKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKy0tLS0tLQ0KIG5ldC9zdW5y cGMveHBydC5jICAgICAgICAgICAgICB8ICAgIDggLQ0KIDkgZmlsZXMgY2hhbmdlZCwgMjgwIGlu c2VydGlvbnMoKyksIDY3IGRlbGV0aW9ucygtKQ0KDQpkaWZmIC11IC0tcmVjdXJzaXZlIC0tbmV3 LWZpbGUgLS1zaG93LWMtZnVuY3Rpb24gbGludXgtMi42LjQtMjMtdW5yYWNlL2ZzL25mcy9uZnM0 c3RhdGUuYyBsaW51eC0yLjYuNC0yNC1ycGNfdGhyb3R0bGUvZnMvbmZzL25mczRzdGF0ZS5jDQot LS0gbGludXgtMi42LjQtMjMtdW5yYWNlL2ZzL25mcy9uZnM0c3RhdGUuYwkyMDA0LTAzLTA0IDE2 OjQzOjA1LjAwMDAwMDAwMCAtMDUwMA0KKysrIGxpbnV4LTIuNi40LTI0LXJwY190aHJvdHRsZS9m cy9uZnMvbmZzNHN0YXRlLmMJMjAwNC0wMy0wNCAxNjo0NTowMS4wMDAwMDAwMDAgLTA1MDANCkBA IC0xMDUsNyArMTA1LDcgQEAgbmZzNF9hbGxvY19jbGllbnQoc3RydWN0IGluX2FkZHIgKmFkZHIp DQogCQlJTklUX1dPUksoJmNscC0+Y2xfcmVuZXdkLCBuZnM0X3JlbmV3X3N0YXRlLCBjbHApOw0K IAkJSU5JVF9MSVNUX0hFQUQoJmNscC0+Y2xfc3VwZXJibG9ja3MpOw0KIAkJaW5pdF93YWl0cXVl dWVfaGVhZCgmY2xwLT5jbF93YWl0cSk7DQotCQlJTklUX1JQQ19XQUlUUSgmY2xwLT5jbF9ycGN3 YWl0cSwgIk5GUzQgY2xpZW50Iik7DQorCQlycGNfaW5pdF93YWl0X3F1ZXVlKCZjbHAtPmNsX3Jw Y3dhaXRxLCAiTkZTNCBjbGllbnQiKTsNCiAJCWNscC0+Y2xfc3RhdGUgPSAxIDw8IE5GUzRDTE5U X05FVzsNCiAJfQ0KIAlyZXR1cm4gY2xwOw0KZGlmZiAtdSAtLXJlY3Vyc2l2ZSAtLW5ldy1maWxl IC0tc2hvdy1jLWZ1bmN0aW9uIGxpbnV4LTIuNi40LTIzLXVucmFjZS9mcy9uZnMvcmVhZC5jIGxp bnV4LTIuNi40LTI0LXJwY190aHJvdHRsZS9mcy9uZnMvcmVhZC5jDQotLS0gbGludXgtMi42LjQt MjMtdW5yYWNlL2ZzL25mcy9yZWFkLmMJMjAwNC0wMy0wNCAxNjo0NDoyMi4wMDAwMDAwMDAgLTA1 MDANCisrKyBsaW51eC0yLjYuNC0yNC1ycGNfdGhyb3R0bGUvZnMvbmZzL3JlYWQuYwkyMDA0LTAz LTA0IDE2OjQ1OjAxLjAwMDAwMDAwMCAtMDUwMA0KQEAgLTIzNCw2ICsyMzQsNyBAQCBzdGF0aWMg dm9pZCBuZnNfcmVhZF9ycGNzZXR1cChzdHJ1Y3QgbmZzDQogDQogCU5GU19QUk9UTyhpbm9kZSkt PnJlYWRfc2V0dXAoZGF0YSk7DQogDQorCWRhdGEtPnRhc2sudGtfY29va2llID0gKHVuc2lnbmVk IGxvbmcpaW5vZGU7DQogCWRhdGEtPnRhc2sudGtfY2FsbGRhdGEgPSBkYXRhOw0KIAkvKiBSZWxl YXNlIHJlcXVlc3RzICovDQogCWRhdGEtPnRhc2sudGtfcmVsZWFzZSA9IG5mc19yZWFkZGF0YV9y ZWxlYXNlOw0KZGlmZiAtdSAtLXJlY3Vyc2l2ZSAtLW5ldy1maWxlIC0tc2hvdy1jLWZ1bmN0aW9u IGxpbnV4LTIuNi40LTIzLXVucmFjZS9mcy9uZnMvd3JpdGUuYyBsaW51eC0yLjYuNC0yNC1ycGNf dGhyb3R0bGUvZnMvbmZzL3dyaXRlLmMNCi0tLSBsaW51eC0yLjYuNC0yMy11bnJhY2UvZnMvbmZz L3dyaXRlLmMJMjAwNC0wMy0wNCAxNjo0NDozOC4wMDAwMDAwMDAgLTA1MDANCisrKyBsaW51eC0y LjYuNC0yNC1ycGNfdGhyb3R0bGUvZnMvbmZzL3dyaXRlLmMJMjAwNC0wMy0wNCAxNzozMjo0OC4w MDAwMDAwMDAgLTA1MDANCkBAIC0xNzMsMTUgKzE3MywxNCBAQCBzdGF0aWMgdm9pZCBuZnNfbWFy a191cHRvZGF0ZShzdHJ1Y3QgcGFnDQogICogV3JpdGUgYSBwYWdlIHN5bmNocm9ub3VzbHkuDQog ICogT2Zmc2V0IGlzIHRoZSBkYXRhIG9mZnNldCB3aXRoaW4gdGhlIHBhZ2UuDQogICovDQotc3Rh dGljIGludA0KLW5mc193cml0ZXBhZ2Vfc3luYyhzdHJ1Y3QgZmlsZSAqZmlsZSwgc3RydWN0IGlu b2RlICppbm9kZSwgc3RydWN0IHBhZ2UgKnBhZ2UsDQotCQkgICB1bnNpZ25lZCBpbnQgb2Zmc2V0 LCB1bnNpZ25lZCBpbnQgY291bnQpDQorc3RhdGljIGludCBuZnNfd3JpdGVwYWdlX3N5bmMoc3Ry dWN0IGZpbGUgKmZpbGUsIHN0cnVjdCBpbm9kZSAqaW5vZGUsDQorCQlzdHJ1Y3QgcGFnZSAqcGFn ZSwgdW5zaWduZWQgaW50IG9mZnNldCwgdW5zaWduZWQgaW50IGNvdW50LA0KKwkJaW50IGhvdykN CiB7DQogCXVuc2lnbmVkIGludAl3c2l6ZSA9IE5GU19TRVJWRVIoaW5vZGUpLT53c2l6ZTsNCiAJ aW50CQlyZXN1bHQsIHdyaXR0ZW4gPSAwOw0KLQlpbnQJCXN3YXBmaWxlID0gSVNfU1dBUEZJTEUo aW5vZGUpOw0KIAlzdHJ1Y3QgbmZzX3dyaXRlX2RhdGEJd2RhdGEgPSB7DQotCQkuZmxhZ3MJCT0g c3dhcGZpbGUgPyBORlNfUlBDX1NXQVBGTEFHUyA6IDAsDQorCQkuZmxhZ3MJCT0gaG93LA0KIAkJ LmNyZWQJCT0gTlVMTCwNCiAJCS5pbm9kZQkJPSBpbm9kZSwNCiAJCS5hcmdzCQk9IHsNCkBAIC0y MDUsNyArMjA0LDcgQEAgbmZzX3dyaXRlcGFnZV9zeW5jKHN0cnVjdCBmaWxlICpmaWxlLCBzdA0K IA0KIAluZnNfYmVnaW5fZGF0YV91cGRhdGUoaW5vZGUpOw0KIAlkbyB7DQotCQlpZiAoY291bnQg PCB3c2l6ZSAmJiAhc3dhcGZpbGUpDQorCQlpZiAoY291bnQgPCB3c2l6ZSkNCiAJCQl3ZGF0YS5h cmdzLmNvdW50ID0gY291bnQ7DQogCQl3ZGF0YS5hcmdzLm9mZnNldCA9IHBhZ2Vfb2Zmc2V0KHBh Z2UpICsgd2RhdGEuYXJncy5wZ2Jhc2U7DQogDQpAQCAtMjY0LDYgKzI2MywxNSBAQCBuZnNfd3Jp dGVwYWdlX2FzeW5jKHN0cnVjdCBmaWxlICpmaWxlLCBzDQogCXJldHVybiBzdGF0dXM7DQogfQ0K IA0KK3N0YXRpYyBpbnQgd2JfcHJpb3JpdHkoc3RydWN0IHdyaXRlYmFja19jb250cm9sICp3YmMp DQorew0KKwlpZiAod2JjLT5mb3JfcmVjbGFpbSkNCisJCXJldHVybiBGTFVTSF9ISUdIUFJJOw0K KwlpZiAod2JjLT5mb3Jfa3VwZGF0ZSkNCisJCXJldHVybiBGTFVTSF9MT1dQUkk7DQorCXJldHVy biAwOw0KK30NCisNCiAvKg0KICAqIFdyaXRlIGFuIG1tYXBwZWQgcGFnZSB0byB0aGUgc2VydmVy Lg0KICAqLw0KQEAgLTI3NCw2ICsyODIsNyBAQCBpbnQgbmZzX3dyaXRlcGFnZShzdHJ1Y3QgcGFn ZSAqcGFnZSwgc3RyDQogCXVuc2lnbmVkIG9mZnNldCA9IFBBR0VfQ0FDSEVfU0laRTsNCiAJbG9m Zl90IGlfc2l6ZSA9IGlfc2l6ZV9yZWFkKGlub2RlKTsNCiAJaW50IGlub2RlX3JlZmVyZW5jZWQg PSAwOw0KKwlpbnQgcHJpb3JpdHkgPSB3Yl9wcmlvcml0eSh3YmMpOw0KIAlpbnQgZXJyOw0KIA0K IAkvKg0KQEAgLTI4OSw3ICsyOTgsNyBAQCBpbnQgbmZzX3dyaXRlcGFnZShzdHJ1Y3QgcGFnZSAq cGFnZSwgc3RyDQogCWVuZF9pbmRleCA9IGlfc2l6ZSA+PiBQQUdFX0NBQ0hFX1NISUZUOw0KIA0K IAkvKiBFbnN1cmUgd2UndmUgZmx1c2hlZCBvdXQgYW55IHByZXZpb3VzIHdyaXRlcyAqLw0KLQlu ZnNfd2JfcGFnZShpbm9kZSxwYWdlKTsNCisJbmZzX3diX3BhZ2VfcHJpb3JpdHkoaW5vZGUsIHBh Z2UsIHByaW9yaXR5KTsNCiANCiAJLyogZWFzeSBjYXNlICovDQogCWlmIChwYWdlLT5pbmRleCA8 IGVuZF9pbmRleCkNCkBAIC0zMTEsNyArMzIwLDcgQEAgZG9faXQ6DQogCQkJCWVyciA9IFdSSVRF UEFHRV9BQ1RJVkFURTsNCiAJCX0NCiAJfSBlbHNlIHsNCi0JCWVyciA9IG5mc193cml0ZXBhZ2Vf c3luYyhOVUxMLCBpbm9kZSwgcGFnZSwgMCwgb2Zmc2V0KTsgDQorCQllcnIgPSBuZnNfd3JpdGVw YWdlX3N5bmMoTlVMTCwgaW5vZGUsIHBhZ2UsIDAsIG9mZnNldCwgcHJpb3JpdHkpOyANCiAJCWlm IChlcnIgPT0gb2Zmc2V0KQ0KIAkJCWVyciA9IDA7DQogCX0NCkBAIC0zNDIsNyArMzUxLDcgQEAg aW50IG5mc193cml0ZXBhZ2VzKHN0cnVjdCBhZGRyZXNzX3NwYWNlIA0KIAkJCXJldHVybiAwOw0K IAkJbmZzX3dhaXRfb25fd3JpdGVfY29uZ2VzdGlvbihtYXBwaW5nLCAwKTsNCiAJfQ0KLQllcnIg PSBuZnNfZmx1c2hfaW5vZGUoaW5vZGUsIDAsIDAsIDApOw0KKwllcnIgPSBuZnNfZmx1c2hfaW5v ZGUoaW5vZGUsIDAsIDAsIHdiX3ByaW9yaXR5KHdiYykpOw0KIAlpZiAoZXJyIDwgMCkNCiAJCWdv dG8gb3V0Ow0KIAl3YmMtPm5yX3RvX3dyaXRlIC09IGVycjsNCkBAIC0zNTEsNyArMzYwLDcgQEAg aW50IG5mc193cml0ZXBhZ2VzKHN0cnVjdCBhZGRyZXNzX3NwYWNlIA0KIAkJaWYgKGVyciA8IDAp DQogCQkJZ290byBvdXQ7DQogCX0NCi0JZXJyID0gbmZzX2NvbW1pdF9pbm9kZShpbm9kZSwgMCwg MCwgMCk7DQorCWVyciA9IG5mc19jb21taXRfaW5vZGUoaW5vZGUsIDAsIDAsIHdiX3ByaW9yaXR5 KHdiYykpOw0KIAlpZiAoZXJyID4gMCkNCiAJCXdiYy0+bnJfdG9fd3JpdGUgLT0gZXJyOw0KIG91 dDoNCkBAIC03MTYsOCArNzI1LDggQEAgbmZzX2ZsdXNoX2luY29tcGF0aWJsZShzdHJ1Y3QgZmls ZSAqZmlsZQ0KICAqIFhYWDogS2VlcCBhbiBleWUgb24gZ2VuZXJpY19maWxlX3JlYWQgdG8gbWFr ZSBzdXJlIGl0IGRvZXNuJ3QgZG8gYmFkDQogICogdGhpbmdzIHdpdGggYSBwYWdlIHNjaGVkdWxl ZCBmb3IgYW4gUlBDIGNhbGwgKGUuZy4gaW52YWxpZGF0ZSBpdCkuDQogICovDQotaW50DQotbmZz X3VwZGF0ZXBhZ2Uoc3RydWN0IGZpbGUgKmZpbGUsIHN0cnVjdCBwYWdlICpwYWdlLCB1bnNpZ25l ZCBpbnQgb2Zmc2V0LCB1bnNpZ25lZCBpbnQgY291bnQpDQoraW50IG5mc191cGRhdGVwYWdlKHN0 cnVjdCBmaWxlICpmaWxlLCBzdHJ1Y3QgcGFnZSAqcGFnZSwNCisJCXVuc2lnbmVkIGludCBvZmZz ZXQsIHVuc2lnbmVkIGludCBjb3VudCkNCiB7DQogCXN0cnVjdCBkZW50cnkJKmRlbnRyeSA9IGZp bGUtPmZfZGVudHJ5Ow0KIAlzdHJ1Y3QgaW5vZGUJKmlub2RlID0gcGFnZS0+bWFwcGluZy0+aG9z dDsNCkBAIC03MjksNyArNzM4LDcgQEAgbmZzX3VwZGF0ZXBhZ2Uoc3RydWN0IGZpbGUgKmZpbGUs IHN0cnVjdA0KIAkJY291bnQsIChsb25nIGxvbmcpKHBhZ2Vfb2Zmc2V0KHBhZ2UpICtvZmZzZXQp KTsNCiANCiAJaWYgKElTX1NZTkMoaW5vZGUpKSB7DQotCQlzdGF0dXMgPSBuZnNfd3JpdGVwYWdl X3N5bmMoZmlsZSwgaW5vZGUsIHBhZ2UsIG9mZnNldCwgY291bnQpOw0KKwkJc3RhdHVzID0gbmZz X3dyaXRlcGFnZV9zeW5jKGZpbGUsIGlub2RlLCBwYWdlLCBvZmZzZXQsIGNvdW50LCAwKTsNCiAJ CWlmIChzdGF0dXMgPiAwKSB7DQogCQkJaWYgKG9mZnNldCA9PSAwICYmIHN0YXR1cyA9PSBQQUdF X0NBQ0hFX1NJWkUpDQogCQkJCVNldFBhZ2VVcHRvZGF0ZShwYWdlKTsNCkBAIC04MjAsNiArODI5 LDE3IEBAIG91dDoNCiAJbmZzX3VubG9ja19yZXF1ZXN0KHJlcSk7DQogfQ0KIA0KK3N0YXRpYyBp bmxpbmUgaW50IGZsdXNoX3Rhc2tfcHJpb3JpdHkoaW50IGhvdykNCit7DQorCXN3aXRjaCAoaG93 ICYgKEZMVVNIX0hJR0hQUkl8RkxVU0hfTE9XUFJJKSkgew0KKwkJY2FzZSBGTFVTSF9ISUdIUFJJ Og0KKwkJCXJldHVybiBSUENfUFJJT1JJVFlfSElHSDsNCisJCWNhc2UgRkxVU0hfTE9XUFJJOg0K KwkJCXJldHVybiBSUENfUFJJT1JJVFlfTE9XOw0KKwl9DQorCXJldHVybiBSUENfUFJJT1JJVFlf Tk9STUFMOw0KK30NCisNCiAvKg0KICAqIFNldCB1cCB0aGUgYXJndW1lbnQvcmVzdWx0IHN0b3Jh Z2UgcmVxdWlyZWQgZm9yIHRoZSBSUEMgY2FsbC4NCiAgKi8NCkBAIC04NDksNiArODY5LDggQEAg c3RhdGljIHZvaWQgbmZzX3dyaXRlX3JwY3NldHVwKHN0cnVjdCBuZg0KIA0KIAlORlNfUFJPVE8o aW5vZGUpLT53cml0ZV9zZXR1cChkYXRhLCBob3cpOw0KIA0KKwlkYXRhLT50YXNrLnRrX3ByaW9y aXR5ID0gZmx1c2hfdGFza19wcmlvcml0eShob3cpOw0KKwlkYXRhLT50YXNrLnRrX2Nvb2tpZSA9 ICh1bnNpZ25lZCBsb25nKWlub2RlOw0KIAlkYXRhLT50YXNrLnRrX2NhbGxkYXRhID0gZGF0YTsN CiAJLyogUmVsZWFzZSByZXF1ZXN0cyAqLw0KIAlkYXRhLT50YXNrLnRrX3JlbGVhc2UgPSBuZnNf d3JpdGVkYXRhX3JlbGVhc2U7DQpAQCAtMTIxMCw2ICsxMjMyLDggQEAgc3RhdGljIHZvaWQgbmZz X2NvbW1pdF9ycGNzZXR1cChzdHJ1Y3QgbA0KIAkNCiAJTkZTX1BST1RPKGlub2RlKS0+Y29tbWl0 X3NldHVwKGRhdGEsIGhvdyk7DQogDQorCWRhdGEtPnRhc2sudGtfcHJpb3JpdHkgPSBmbHVzaF90 YXNrX3ByaW9yaXR5KGhvdyk7DQorCWRhdGEtPnRhc2sudGtfY29va2llID0gKHVuc2lnbmVkIGxv bmcpaW5vZGU7DQogCWRhdGEtPnRhc2sudGtfY2FsbGRhdGEgPSBkYXRhOw0KIAkvKiBSZWxlYXNl IHJlcXVlc3RzICovDQogCWRhdGEtPnRhc2sudGtfcmVsZWFzZSA9IG5mc19jb21taXRfcmVsZWFz ZTsNCmRpZmYgLXUgLS1yZWN1cnNpdmUgLS1uZXctZmlsZSAtLXNob3ctYy1mdW5jdGlvbiBsaW51 eC0yLjYuNC0yMy11bnJhY2UvaW5jbHVkZS9saW51eC9uZnNfZnMuaCBsaW51eC0yLjYuNC0yNC1y cGNfdGhyb3R0bGUvaW5jbHVkZS9saW51eC9uZnNfZnMuaA0KLS0tIGxpbnV4LTIuNi40LTIzLXVu cmFjZS9pbmNsdWRlL2xpbnV4L25mc19mcy5oCTIwMDQtMDMtMDQgMTY6NDQ6MzEuMDAwMDAwMDAw IC0wNTAwDQorKysgbGludXgtMi42LjQtMjQtcnBjX3Rocm90dGxlL2luY2x1ZGUvbGludXgvbmZz X2ZzLmgJMjAwNC0wMy0wNCAxNjo0NTowMS4wMDAwMDAwMDAgLTA1MDANCkBAIC02OSw2ICs2OSw4 IEBADQogI2RlZmluZSBGTFVTSF9TWU5DCQkxCS8qIGZpbGUgYmVpbmcgc3luY2VkLCBvciBjb250 ZW50aW9uICovDQogI2RlZmluZSBGTFVTSF9XQUlUCQkyCS8qIHdhaXQgZm9yIGNvbXBsZXRpb24g Ki8NCiAjZGVmaW5lIEZMVVNIX1NUQUJMRQkJNAkvKiBjb21taXQgdG8gc3RhYmxlIHN0b3JhZ2Ug Ki8NCisjZGVmaW5lIEZMVVNIX0xPV1BSSQkJOAkvKiBsb3cgcHJpb3JpdHkgYmFja2dyb3VuZCBm bHVzaCAqLw0KKyNkZWZpbmUgRkxVU0hfSElHSFBSSQkJMTYJLyogaGlnaCBwcmlvcml0eSBtZW1v cnkgcmVjbGFpbSBmbHVzaCAqLw0KIA0KICNpZmRlZiBfX0tFUk5FTF9fDQogDQpAQCAtMzcxLDE0 ICszNzMsMTggQEAgbmZzX3diX2FsbChzdHJ1Y3QgaW5vZGUgKmlub2RlKQ0KIC8qDQogICogV3Jp dGUgYmFjayBhbGwgcmVxdWVzdHMgb24gb25lIHBhZ2UgLSB3ZSBkbyB0aGlzIGJlZm9yZSByZWFk aW5nIGl0Lg0KICAqLw0KLXN0YXRpYyBpbmxpbmUgaW50DQotbmZzX3diX3BhZ2Uoc3RydWN0IGlu b2RlICppbm9kZSwgc3RydWN0IHBhZ2UqIHBhZ2UpDQorc3RhdGljIGlubGluZSBpbnQgbmZzX3di X3BhZ2VfcHJpb3JpdHkoc3RydWN0IGlub2RlICppbm9kZSwgc3RydWN0IHBhZ2UqIHBhZ2UsIGlu dCBob3cpDQogew0KIAlpbnQgZXJyb3IgPSBuZnNfc3luY19pbm9kZShpbm9kZSwgcGFnZS0+aW5k ZXgsIDEsDQotCQkJCQkJRkxVU0hfV0FJVCB8IEZMVVNIX1NUQUJMRSk7DQorCQkJaG93IHwgRkxV U0hfV0FJVCB8IEZMVVNIX1NUQUJMRSk7DQogCXJldHVybiAoZXJyb3IgPCAwKSA/IGVycm9yIDog MDsNCiB9DQogDQorc3RhdGljIGlubGluZSBpbnQgbmZzX3diX3BhZ2Uoc3RydWN0IGlub2RlICpp bm9kZSwgc3RydWN0IHBhZ2UqIHBhZ2UpDQorew0KKwlyZXR1cm4gbmZzX3diX3BhZ2VfcHJpb3Jp dHkoaW5vZGUsIHBhZ2UsIDApOw0KK30NCisNCiAvKiBIYWNrIGZvciBmdXR1cmUgTkZTIHN3YXAg c3VwcG9ydCAqLw0KICNpZm5kZWYgSVNfU1dBUEZJTEUNCiAjIGRlZmluZSBJU19TV0FQRklMRShp bm9kZSkJKDApDQpkaWZmIC11IC0tcmVjdXJzaXZlIC0tbmV3LWZpbGUgLS1zaG93LWMtZnVuY3Rp b24gbGludXgtMi42LjQtMjMtdW5yYWNlL2luY2x1ZGUvbGludXgvc3VucnBjL3NjaGVkLmggbGlu dXgtMi42LjQtMjQtcnBjX3Rocm90dGxlL2luY2x1ZGUvbGludXgvc3VucnBjL3NjaGVkLmgNCi0t LSBsaW51eC0yLjYuNC0yMy11bnJhY2UvaW5jbHVkZS9saW51eC9zdW5ycGMvc2NoZWQuaAkyMDA0 LTAzLTA0IDE2OjIwOjU4LjAwMDAwMDAwMCAtMDUwMA0KKysrIGxpbnV4LTIuNi40LTI0LXJwY190 aHJvdHRsZS9pbmNsdWRlL2xpbnV4L3N1bnJwYy9zY2hlZC5oCTIwMDQtMDMtMDQgMTY6NDU6MDIu MDAwMDAwMDAwIC0wNTAwDQpAQCAtNDksNiArNDksOCBAQCBzdHJ1Y3QgcnBjX3Rhc2sgew0KIAkJ CQl0a19jcmVkX3JldHJ5LA0KIAkJCQl0a19zdWlkX3JldHJ5Ow0KIA0KKwl1bnNpZ25lZCBsb25n CQl0a19jb29raWU7CS8qIENvb2tpZSBmb3IgYmF0Y2hpbmcgdGFza3MgKi8NCisNCiAJLyoNCiAJ ICogdGltZW91dF9mbiAgIHRvIGJlIGV4ZWN1dGVkIGJ5IHRpbWVyIGJvdHRvbSBoYWxmDQogCSAq IGNhbGxiYWNrCXRvIGJlIGV4ZWN1dGVkIGFmdGVyIHdha2luZyB1cA0KQEAgLTcyLDcgKzc0LDkg QEAgc3RydWN0IHJwY190YXNrIHsNCiAJdW5zaWduZWQgbG9uZwkJdGtfdGltZW91dDsJLyogdGlt ZW91dCBmb3IgcnBjX3NsZWVwKCkgKi8NCiAJdW5zaWduZWQgc2hvcnQJCXRrX2ZsYWdzOwkvKiBt aXNjIGZsYWdzICovDQogCXVuc2lnbmVkIGNoYXIJCXRrX2FjdGl2ZSAgIDogMTsvKiBUYXNrIGhh cyBiZWVuIGFjdGl2YXRlZCAqLw0KKwl1bnNpZ25lZCBjaGFyCQl0a19wcmlvcml0eSA6IDI7Lyog VGFzayBwcmlvcml0eSAqLw0KIAl1bnNpZ25lZCBsb25nCQl0a19ydW5zdGF0ZTsJLyogVGFzayBy dW4gc3RhdHVzICovDQorCXN0cnVjdCBsaXN0X2hlYWQJdGtfbGlua3M7CS8qIGxpbmtzIHRvIHJl bGF0ZWQgdGFza3MgKi8NCiAjaWZkZWYgUlBDX0RFQlVHDQogCXVuc2lnbmVkIHNob3J0CQl0a19w aWQ7CQkvKiBkZWJ1Z2dpbmcgYWlkICovDQogI2VuZGlmDQpAQCAtMTM4LDI4ICsxNDIsNTggQEAg dHlwZWRlZiB2b2lkCQkJKCpycGNfYWN0aW9uKShzdHJ1Y3QgcnBjXw0KIAl9IHdoaWxlKDApDQog DQogLyoNCisgKiBUYXNrIHByaW9yaXRpZXMuDQorICogTm90ZTogaWYgeW91IGNoYW5nZSB0aGVz ZSwgeW91IG11c3QgYWxzbyBjaGFuZ2UNCisgKiB0aGUgdGFzayBpbml0aWFsaXphdGlvbiBkZWZp bml0aW9ucyBiZWxvdy4NCisgKi8NCisjZGVmaW5lIFJQQ19QUklPUklUWV9MT1cJMA0KKyNkZWZp bmUgUlBDX1BSSU9SSVRZX05PUk1BTAkxDQorI2RlZmluZSBSUENfUFJJT1JJVFlfSElHSAkyDQor I2RlZmluZSBSUENfTlJfUFJJT1JJVFkJCShSUENfUFJJT1JJVFlfSElHSCsxKQ0KKw0KKy8qDQog ICogUlBDIHN5bmNocm9uaXphdGlvbiBvYmplY3RzDQogICovDQogc3RydWN0IHJwY193YWl0X3F1 ZXVlIHsNCi0Jc3RydWN0IGxpc3RfaGVhZAl0YXNrczsNCisJc3RydWN0IGxpc3RfaGVhZAl0YXNr c1tSUENfTlJfUFJJT1JJVFldOwkvKiB0YXNrIHF1ZXVlIGZvciBlYWNoIHByaW9yaXR5IGxldmVs ICovDQorCXVuc2lnbmVkIGxvbmcJCWNvb2tpZTsJCQkvKiBjb29raWUgb2YgbGFzdCB0YXNrIHNl cnZpY2VkICovDQorCXVuc2lnbmVkIGNoYXIJCW1heHByaW9yaXR5OwkJLyogbWF4aW11bSBwcmlv cml0eSAoMCBpZiBxdWV1ZSBpcyBub3QgYSBwcmlvcml0eSBxdWV1ZSkgKi8NCisJdW5zaWduZWQg Y2hhcgkJcHJpb3JpdHk7CQkvKiBjdXJyZW50IHByaW9yaXR5ICovDQorCXVuc2lnbmVkIGNoYXIJ CWNvdW50OwkJCS8qICMgdGFzayBncm91cHMgcmVtYWluaW5nIHNlcnZpY2VkIHNvIGZhciAqLw0K Kwl1bnNpZ25lZCBjaGFyCQlucjsJCQkvKiAjIHRhc2tzIHJlbWFpbmluZyBmb3IgY29va2llICov DQogI2lmZGVmIFJQQ19ERUJVRw0KLQljaGFyICoJCQluYW1lOw0KKwljb25zdCBjaGFyICoJCW5h bWU7DQogI2VuZGlmDQogfTsNCiANCisvKg0KKyAqIFRoaXMgaXMgdGhlICMgcmVxdWVzdHMgdG8g c2VuZCBjb25zZWN1dGl2ZWx5DQorICogZnJvbSBhIHNpbmdsZSBjb29raWUuICBUaGUgYWltIGlz IHRvIGltcHJvdmUNCisgKiBwZXJmb3JtYW5jZSBvZiBORlMgb3BlcmF0aW9ucyBzdWNoIGFzIHJl YWQvd3JpdGUuDQorICovDQorI2RlZmluZSBSUENfQkFUQ0hfQ09VTlQJCQkxNg0KKw0KICNpZm5k ZWYgUlBDX0RFQlVHDQotIyBkZWZpbmUgUlBDX1dBSVRRX0lOSVQodmFyLHFuYW1lKSAoKHN0cnVj dCBycGNfd2FpdF9xdWV1ZSkge0xJU1RfSEVBRF9JTklUKHZhcil9KQ0KLSMgZGVmaW5lIFJQQ19X QUlUUSh2YXIscW5hbWUpICAgICAgc3RydWN0IHJwY193YWl0X3F1ZXVlIHZhciA9IFJQQ19XQUlU UV9JTklUKHZhci50YXNrcyxxbmFtZSkNCi0jIGRlZmluZSBJTklUX1JQQ19XQUlUUShwdHIscW5h bWUpIGRvIHsgXA0KLQlJTklUX0xJU1RfSEVBRCgmKHB0ciktPnRhc2tzKTsgXA0KLQl9IHdoaWxl KDApDQorIyBkZWZpbmUgUlBDX1dBSVRRX0lOSVQodmFyLHFuYW1lKSB7IFwNCisJCS50YXNrcyA9 IHsgXA0KKwkJCVswXSA9IExJU1RfSEVBRF9JTklUKHZhci50YXNrc1swXSksIFwNCisJCQlbMV0g PSBMSVNUX0hFQURfSU5JVCh2YXIudGFza3NbMV0pLCBcDQorCQkJWzJdID0gTElTVF9IRUFEX0lO SVQodmFyLnRhc2tzWzJdKSwgXA0KKwkJfSwgXA0KKwl9DQogI2Vsc2UNCi0jIGRlZmluZSBSUENf V0FJVFFfSU5JVCh2YXIscW5hbWUpICgoc3RydWN0IHJwY193YWl0X3F1ZXVlKSB7TElTVF9IRUFE X0lOSVQodmFyLnRhc2tzKSwgcW5hbWV9KQ0KLSMgZGVmaW5lIFJQQ19XQUlUUSh2YXIscW5hbWUp ICAgICAgc3RydWN0IHJwY193YWl0X3F1ZXVlIHZhciA9IFJQQ19XQUlUUV9JTklUKHZhcixxbmFt ZSkNCi0jIGRlZmluZSBJTklUX1JQQ19XQUlUUShwdHIscW5hbWUpIGRvIHsgXA0KLQlJTklUX0xJ U1RfSEVBRCgmKHB0ciktPnRhc2tzKTsgKHB0ciktPm5hbWUgPSBxbmFtZTsgXA0KLQl9IHdoaWxl KDApDQorIyBkZWZpbmUgUlBDX1dBSVRRX0lOSVQodmFyLHFuYW1lKSB7IFwNCisJCS50YXNrcyA9 IHsgXA0KKwkJCVswXSA9IExJU1RfSEVBRF9JTklUKHZhci50YXNrc1swXSksIFwNCisJCQlbMV0g PSBMSVNUX0hFQURfSU5JVCh2YXIudGFza3NbMV0pLCBcDQorCQkJWzJdID0gTElTVF9IRUFEX0lO SVQodmFyLnRhc2tzWzJdKSwgXA0KKwkJfSwgXA0KKwkJLm5hbWUgPSBxbmFtZSwgXA0KKwl9DQog I2VuZGlmDQorIyBkZWZpbmUgUlBDX1dBSVRRKHZhcixxbmFtZSkgICAgICBzdHJ1Y3QgcnBjX3dh aXRfcXVldWUgdmFyID0gUlBDX1dBSVRRX0lOSVQodmFyLHFuYW1lKQ0KKw0KKyNkZWZpbmUgUlBD X0lTX1BSSU9SSVRZKHEpCQkoKHEpLT5tYXhwcmlvcml0eSA+IDApDQogDQogLyoNCiAgKiBGdW5j dGlvbiBwcm90b3R5cGVzDQpAQCAtMTc1LDYgKzIwOSw4IEBAIHZvaWQJCXJwY19ydW5fY2hpbGQo c3RydWN0IHJwY190YXNrICpwYXINCiAJCQkJCXJwY19hY3Rpb24gYWN0aW9uKTsNCiBpbnQJCXJw Y19hZGRfd2FpdF9xdWV1ZShzdHJ1Y3QgcnBjX3dhaXRfcXVldWUgKiwgc3RydWN0IHJwY190YXNr ICopOw0KIHZvaWQJCXJwY19yZW1vdmVfd2FpdF9xdWV1ZShzdHJ1Y3QgcnBjX3Rhc2sgKik7DQor dm9pZAkJcnBjX2luaXRfcHJpb3JpdHlfd2FpdF9xdWV1ZShzdHJ1Y3QgcnBjX3dhaXRfcXVldWUg KiwgY29uc3QgY2hhciAqKTsNCit2b2lkCQlycGNfaW5pdF93YWl0X3F1ZXVlKHN0cnVjdCBycGNf d2FpdF9xdWV1ZSAqLCBjb25zdCBjaGFyICopOw0KIHZvaWQJCXJwY19zbGVlcF9vbihzdHJ1Y3Qg cnBjX3dhaXRfcXVldWUgKiwgc3RydWN0IHJwY190YXNrICosDQogCQkJCQlycGNfYWN0aW9uIGFj dGlvbiwgcnBjX2FjdGlvbiB0aW1lcik7DQogdm9pZAkJcnBjX2FkZF90aW1lcihzdHJ1Y3QgcnBj X3Rhc2sgKiwgcnBjX2FjdGlvbik7DQpAQCAtMTk0LDE2ICsyMzAsMTQgQEAgdm9pZAkJcnBjX3No b3dfdGFza3Modm9pZCk7DQogaW50CQlycGNfaW5pdF9tZW1wb29sKHZvaWQpOw0KIHZvaWQJCXJw Y19kZXN0cm95X21lbXBvb2wodm9pZCk7DQogDQotc3RhdGljIF9faW5saW5lX18gdm9pZA0KLXJw Y19leGl0KHN0cnVjdCBycGNfdGFzayAqdGFzaywgaW50IHN0YXR1cykNCitzdGF0aWMgaW5saW5l IHZvaWQgcnBjX2V4aXQoc3RydWN0IHJwY190YXNrICp0YXNrLCBpbnQgc3RhdHVzKQ0KIHsNCiAJ dGFzay0+dGtfc3RhdHVzID0gc3RhdHVzOw0KIAl0YXNrLT50a19hY3Rpb24gPSBOVUxMOw0KIH0N CiANCiAjaWZkZWYgUlBDX0RFQlVHDQotc3RhdGljIF9faW5saW5lX18gY2hhciAqDQotcnBjX3Fu YW1lKHN0cnVjdCBycGNfd2FpdF9xdWV1ZSAqcSkNCitzdGF0aWMgaW5saW5lIGNvbnN0IGNoYXIg KiBycGNfcW5hbWUoc3RydWN0IHJwY193YWl0X3F1ZXVlICpxKQ0KIHsNCiAJcmV0dXJuICgocSAm JiBxLT5uYW1lKSA/IHEtPm5hbWUgOiAidW5rbm93biIpOw0KIH0NCmRpZmYgLXUgLS1yZWN1cnNp dmUgLS1uZXctZmlsZSAtLXNob3ctYy1mdW5jdGlvbiBsaW51eC0yLjYuNC0yMy11bnJhY2UvbmV0 L3N1bnJwYy9hdXRoX2dzcy9hdXRoX2dzcy5jIGxpbnV4LTIuNi40LTI0LXJwY190aHJvdHRsZS9u ZXQvc3VucnBjL2F1dGhfZ3NzL2F1dGhfZ3NzLmMNCi0tLSBsaW51eC0yLjYuNC0yMy11bnJhY2Uv bmV0L3N1bnJwYy9hdXRoX2dzcy9hdXRoX2dzcy5jCTIwMDQtMDMtMDQgMTY6MTQ6NTAuMDAwMDAw MDAwIC0wNTAwDQorKysgbGludXgtMi42LjQtMjQtcnBjX3Rocm90dGxlL25ldC9zdW5ycGMvYXV0 aF9nc3MvYXV0aF9nc3MuYwkyMDA0LTAzLTA0IDE2OjQ1OjAyLjAwMDAwMDAwMCAtMDUwMA0KQEAg LTM2NSw3ICszNjUsNyBAQCByZXRyeToNCiAJZ3NzX21zZyA9IGdzc19uZXc7DQogCW1lbXNldChn c3NfbmV3LCAwLCBzaXplb2YoKmdzc19uZXcpKTsNCiAJSU5JVF9MSVNUX0hFQUQoJmdzc19uZXct Pmxpc3QpOw0KLQlJTklUX1JQQ19XQUlUUSgmZ3NzX25ldy0+d2FpdHEsICJSUENTRUNfR1NTIHVw Y2FsbCB3YWl0cSIpOw0KKwlycGNfaW5pdF93YWl0X3F1ZXVlKCZnc3NfbmV3LT53YWl0cSwgIlJQ Q1NFQ19HU1MgdXBjYWxsIHdhaXRxIik7DQogCWF0b21pY19zZXQoJmdzc19uZXctPmNvdW50LCAy KTsNCiAJbXNnID0gJmdzc19uZXctPm1zZzsNCiAJbXNnLT5kYXRhID0gJmdzc19uZXctPnVpZDsN CmRpZmYgLXUgLS1yZWN1cnNpdmUgLS1uZXctZmlsZSAtLXNob3ctYy1mdW5jdGlvbiBsaW51eC0y LjYuNC0yMy11bnJhY2UvbmV0L3N1bnJwYy9jbG50LmMgbGludXgtMi42LjQtMjQtcnBjX3Rocm90 dGxlL25ldC9zdW5ycGMvY2xudC5jDQotLS0gbGludXgtMi42LjQtMjMtdW5yYWNlL25ldC9zdW5y cGMvY2xudC5jCTIwMDQtMDMtMDQgMTY6NDM6NDcuMDAwMDAwMDAwIC0wNTAwDQorKysgbGludXgt Mi42LjQtMjQtcnBjX3Rocm90dGxlL25ldC9zdW5ycGMvY2xudC5jCTIwMDQtMDMtMDQgMTY6NDU6 MDIuMDAwMDAwMDAwIC0wNTAwDQpAQCAtMTQ0LDcgKzE0NCw3IEBAIHJwY19jcmVhdGVfY2xpZW50 KHN0cnVjdCBycGNfeHBydCAqeHBydCwNCiAJY2xudC0+Y2xfdmVycyAgICAgPSB2ZXJzaW9uLT5u dW1iZXI7DQogCWNsbnQtPmNsX3Byb3QgICAgID0geHBydC0+cHJvdDsNCiAJY2xudC0+Y2xfc3Rh dHMgICAgPSBwcm9ncmFtLT5zdGF0czsNCi0JSU5JVF9SUENfV0FJVFEoJmNsbnQtPmNsX3BtYXBf ZGVmYXVsdC5wbV9iaW5kd2FpdCwgImJpbmR3YWl0Iik7DQorCXJwY19pbml0X3dhaXRfcXVldWUo JmNsbnQtPmNsX3BtYXBfZGVmYXVsdC5wbV9iaW5kd2FpdCwgImJpbmR3YWl0Iik7DQogDQogCWlm ICghY2xudC0+Y2xfcG9ydCkNCiAJCWNsbnQtPmNsX2F1dG9iaW5kID0gMTsNCmRpZmYgLXUgLS1y ZWN1cnNpdmUgLS1uZXctZmlsZSAtLXNob3ctYy1mdW5jdGlvbiBsaW51eC0yLjYuNC0yMy11bnJh Y2UvbmV0L3N1bnJwYy9zY2hlZC5jIGxpbnV4LTIuNi40LTI0LXJwY190aHJvdHRsZS9uZXQvc3Vu cnBjL3NjaGVkLmMNCi0tLSBsaW51eC0yLjYuNC0yMy11bnJhY2UvbmV0L3N1bnJwYy9zY2hlZC5j CTIwMDQtMDMtMDQgMTY6NDQ6NTIuMDAwMDAwMDAwIC0wNTAwDQorKysgbGludXgtMi42LjQtMjQt cnBjX3Rocm90dGxlL25ldC9zdW5ycGMvc2NoZWQuYwkyMDA0LTAzLTA0IDE2OjQ1OjAyLjAwMDAw MDAwMCAtMDUwMA0KQEAgLTE2Miw2ICsxNjIsMjYgQEAgcnBjX2RlbGV0ZV90aW1lcihzdHJ1Y3Qg cnBjX3Rhc2sgKnRhc2spDQogfQ0KIA0KIC8qDQorICogQWRkIG5ldyByZXF1ZXN0IHRvIGEgcHJp b3JpdHkgcXVldWUuDQorICovDQorc3RhdGljIHZvaWQgX19ycGNfYWRkX3dhaXRfcXVldWVfcHJp b3JpdHkoc3RydWN0IHJwY193YWl0X3F1ZXVlICpxdWV1ZSwgc3RydWN0IHJwY190YXNrICp0YXNr KQ0KK3sNCisJc3RydWN0IGxpc3RfaGVhZCAqcTsNCisJc3RydWN0IHJwY190YXNrICp0Ow0KKw0K KwlxID0gJnF1ZXVlLT50YXNrc1t0YXNrLT50a19wcmlvcml0eV07DQorCWlmICh1bmxpa2VseSh0 YXNrLT50a19wcmlvcml0eSA+IHF1ZXVlLT5tYXhwcmlvcml0eSkpDQorCQlxID0gJnF1ZXVlLT50 YXNrc1txdWV1ZS0+bWF4cHJpb3JpdHldOw0KKwlsaXN0X2Zvcl9lYWNoX2VudHJ5KHQsIHEsIHRr X2xpc3QpIHsNCisJCWlmICh0LT50a19jb29raWUgPT0gdGFzay0+dGtfY29va2llKSB7DQorCQkJ bGlzdF9hZGRfdGFpbCgmdGFzay0+dGtfbGlzdCwgJnQtPnRrX2xpbmtzKTsNCisJCQlyZXR1cm47 DQorCQl9DQorCX0NCisJbGlzdF9hZGRfdGFpbCgmdGFzay0+dGtfbGlzdCwgcSk7DQorfQ0KKw0K Ky8qDQogICogQWRkIG5ldyByZXF1ZXN0IHRvIHdhaXQgcXVldWUuDQogICoNCiAgKiBTd2FwcGVy IHRhc2tzIGFsd2F5cyBnZXQgaW5zZXJ0ZWQgYXQgdGhlIGhlYWQgb2YgdGhlIHF1ZXVlLg0KQEAg LTE2OSw4ICsxODksNyBAQCBycGNfZGVsZXRlX3RpbWVyKHN0cnVjdCBycGNfdGFzayAqdGFzaykN CiAgKiBpbXByb3ZlIG92ZXJhbGwgcGVyZm9ybWFuY2UuDQogICogRXZlcnlvbmUgZWxzZSBnZXRz IGFwcGVuZGVkIHRvIHRoZSBxdWV1ZSB0byBlbnN1cmUgcHJvcGVyIEZJRk8gYmVoYXZpb3IuDQog ICovDQotc3RhdGljIGlubGluZSBpbnQNCi1fX3JwY19hZGRfd2FpdF9xdWV1ZShzdHJ1Y3QgcnBj X3dhaXRfcXVldWUgKnF1ZXVlLCBzdHJ1Y3QgcnBjX3Rhc2sgKnRhc2spDQorc3RhdGljIGludCBf X3JwY19hZGRfd2FpdF9xdWV1ZShzdHJ1Y3QgcnBjX3dhaXRfcXVldWUgKnF1ZXVlLCBzdHJ1Y3Qg cnBjX3Rhc2sgKnRhc2spDQogew0KIAlpZiAodGFzay0+dGtfcnBjd2FpdCA9PSBxdWV1ZSkNCiAJ CXJldHVybiAwOw0KQEAgLTE3OSwxMCArMTk4LDEyIEBAIF9fcnBjX2FkZF93YWl0X3F1ZXVlKHN0 cnVjdCBycGNfd2FpdF9xdWUNCiAJCXByaW50ayhLRVJOX1dBUk5JTkcgIlJQQzogZG91Ymx5IGVu cXVldWVkIHRhc2shXG4iKTsNCiAJCXJldHVybiAtRVdPVUxEQkxPQ0s7DQogCX0NCi0JaWYgKFJQ Q19JU19TV0FQUEVSKHRhc2spKQ0KLQkJbGlzdF9hZGQoJnRhc2stPnRrX2xpc3QsICZxdWV1ZS0+ dGFza3MpOw0KKwlpZiAoUlBDX0lTX1BSSU9SSVRZKHF1ZXVlKSkNCisJCV9fcnBjX2FkZF93YWl0 X3F1ZXVlX3ByaW9yaXR5KHF1ZXVlLCB0YXNrKTsNCisJZWxzZSBpZiAoUlBDX0lTX1NXQVBQRVIo dGFzaykpDQorCQlsaXN0X2FkZCgmdGFzay0+dGtfbGlzdCwgJnF1ZXVlLT50YXNrc1swXSk7DQog CWVsc2UNCi0JCWxpc3RfYWRkX3RhaWwoJnRhc2stPnRrX2xpc3QsICZxdWV1ZS0+dGFza3MpOw0K KwkJbGlzdF9hZGRfdGFpbCgmdGFzay0+dGtfbGlzdCwgJnF1ZXVlLT50YXNrc1swXSk7DQogCXRh c2stPnRrX3JwY3dhaXQgPSBxdWV1ZTsNCiANCiAJZHByaW50aygiUlBDOiAlNGQgYWRkZWQgdG8g cXVldWUgJXAgXCIlc1wiXG4iLA0KQEAgLTE5MSw4ICsyMTIsNyBAQCBfX3JwY19hZGRfd2FpdF9x dWV1ZShzdHJ1Y3QgcnBjX3dhaXRfcXVlDQogCXJldHVybiAwOw0KIH0NCiANCi1pbnQNCi1ycGNf YWRkX3dhaXRfcXVldWUoc3RydWN0IHJwY193YWl0X3F1ZXVlICpxLCBzdHJ1Y3QgcnBjX3Rhc2sg KnRhc2spDQoraW50IHJwY19hZGRfd2FpdF9xdWV1ZShzdHJ1Y3QgcnBjX3dhaXRfcXVldWUgKnEs IHN0cnVjdCBycGNfdGFzayAqdGFzaykNCiB7DQogCWludAkJcmVzdWx0Ow0KIA0KQEAgLTIwMywx OCArMjIzLDM1IEBAIHJwY19hZGRfd2FpdF9xdWV1ZShzdHJ1Y3QgcnBjX3dhaXRfcXVldWUNCiB9 DQogDQogLyoNCisgKiBSZW1vdmUgcmVxdWVzdCBmcm9tIGEgcHJpb3JpdHkgcXVldWUuDQorICov DQorc3RhdGljIHZvaWQgX19ycGNfcmVtb3ZlX3dhaXRfcXVldWVfcHJpb3JpdHkoc3RydWN0IHJw Y190YXNrICp0YXNrKQ0KK3sNCisJc3RydWN0IHJwY190YXNrICp0Ow0KKw0KKwlpZiAoIWxpc3Rf ZW1wdHkoJnRhc2stPnRrX2xpbmtzKSkgew0KKwkJdCA9IGxpc3RfZW50cnkodGFzay0+dGtfbGlu a3MubmV4dCwgc3RydWN0IHJwY190YXNrLCB0a19saXN0KTsNCisJCWxpc3RfbW92ZSgmdC0+dGtf bGlzdCwgJnRhc2stPnRrX2xpc3QpOw0KKwkJbGlzdF9zcGxpY2VfaW5pdCgmdGFzay0+dGtfbGlu a3MsICZ0LT50a19saW5rcyk7DQorCX0NCisJbGlzdF9kZWwoJnRhc2stPnRrX2xpc3QpOw0KK30N CisNCisvKg0KICAqIFJlbW92ZSByZXF1ZXN0IGZyb20gcXVldWUuDQogICogTm90ZTogbXVzdCBi ZSBjYWxsZWQgd2l0aCBzcGluIGxvY2sgaGVsZC4NCiAgKi8NCi1zdGF0aWMgaW5saW5lIHZvaWQN Ci1fX3JwY19yZW1vdmVfd2FpdF9xdWV1ZShzdHJ1Y3QgcnBjX3Rhc2sgKnRhc2spDQorc3RhdGlj IHZvaWQgX19ycGNfcmVtb3ZlX3dhaXRfcXVldWUoc3RydWN0IHJwY190YXNrICp0YXNrKQ0KIHsN CiAJc3RydWN0IHJwY193YWl0X3F1ZXVlICpxdWV1ZSA9IHRhc2stPnRrX3JwY3dhaXQ7DQogDQog CWlmICghcXVldWUpDQogCQlyZXR1cm47DQogDQotCWxpc3RfZGVsKCZ0YXNrLT50a19saXN0KTsN CisJaWYgKFJQQ19JU19QUklPUklUWShxdWV1ZSkpDQorCQlfX3JwY19yZW1vdmVfd2FpdF9xdWV1 ZV9wcmlvcml0eSh0YXNrKTsNCisJZWxzZQ0KKwkJbGlzdF9kZWwoJnRhc2stPnRrX2xpc3QpOw0K IAl0YXNrLT50a19ycGN3YWl0ID0gTlVMTDsNCiANCiAJZHByaW50aygiUlBDOiAlNGQgcmVtb3Zl ZCBmcm9tIHF1ZXVlICVwIFwiJXNcIlxuIiwNCkBAIC0yMzEsNiArMjY4LDQ4IEBAIHJwY19yZW1v dmVfd2FpdF9xdWV1ZShzdHJ1Y3QgcnBjX3Rhc2sgKnQNCiAJc3Bpbl91bmxvY2tfYmgoJnJwY19x dWV1ZV9sb2NrKTsNCiB9DQogDQorc3RhdGljIGlubGluZSB2b2lkIHJwY19zZXRfd2FpdHF1ZXVl X3ByaW9yaXR5KHN0cnVjdCBycGNfd2FpdF9xdWV1ZSAqcXVldWUsIGludCBwcmlvcml0eSkNCit7 DQorCXF1ZXVlLT5wcmlvcml0eSA9IHByaW9yaXR5Ow0KKwlxdWV1ZS0+Y291bnQgPSAxIDw8IChw cmlvcml0eSAqIDIpOw0KK30NCisNCitzdGF0aWMgaW5saW5lIHZvaWQgcnBjX3NldF93YWl0cXVl dWVfY29va2llKHN0cnVjdCBycGNfd2FpdF9xdWV1ZSAqcXVldWUsIHVuc2lnbmVkIGxvbmcgY29v a2llKQ0KK3sNCisJcXVldWUtPmNvb2tpZSA9IGNvb2tpZTsNCisJcXVldWUtPm5yID0gUlBDX0JB VENIX0NPVU5UOw0KK30NCisNCitzdGF0aWMgaW5saW5lIHZvaWQgcnBjX3Jlc2V0X3dhaXRxdWV1 ZV9wcmlvcml0eShzdHJ1Y3QgcnBjX3dhaXRfcXVldWUgKnF1ZXVlKQ0KK3sNCisJcnBjX3NldF93 YWl0cXVldWVfcHJpb3JpdHkocXVldWUsIHF1ZXVlLT5tYXhwcmlvcml0eSk7DQorCXJwY19zZXRf d2FpdHF1ZXVlX2Nvb2tpZShxdWV1ZSwgMCk7DQorfQ0KKw0KK3N0YXRpYyB2b2lkIF9fcnBjX2lu aXRfcHJpb3JpdHlfd2FpdF9xdWV1ZShzdHJ1Y3QgcnBjX3dhaXRfcXVldWUgKnF1ZXVlLCBjb25z dCBjaGFyICpxbmFtZSwgaW50IG1heHByaW8pDQorew0KKwlpbnQgaTsNCisNCisJZm9yIChpID0g MDsgaSA8IEFSUkFZX1NJWkUocXVldWUtPnRhc2tzKTsgaSsrKQ0KKwkJSU5JVF9MSVNUX0hFQUQo JnF1ZXVlLT50YXNrc1tpXSk7DQorCXF1ZXVlLT5tYXhwcmlvcml0eSA9IG1heHByaW87DQorCXJw Y19yZXNldF93YWl0cXVldWVfcHJpb3JpdHkocXVldWUpOw0KKyNpZmRlZiBSUENfREVCVUcNCisJ cXVldWUtPm5hbWUgPSBxbmFtZTsNCisjZW5kaWYNCit9DQorDQordm9pZCBycGNfaW5pdF9wcmlv cml0eV93YWl0X3F1ZXVlKHN0cnVjdCBycGNfd2FpdF9xdWV1ZSAqcXVldWUsIGNvbnN0IGNoYXIg KnFuYW1lKQ0KK3sNCisJX19ycGNfaW5pdF9wcmlvcml0eV93YWl0X3F1ZXVlKHF1ZXVlLCBxbmFt ZSwgUlBDX1BSSU9SSVRZX0hJR0gpOw0KK30NCisNCit2b2lkIHJwY19pbml0X3dhaXRfcXVldWUo c3RydWN0IHJwY193YWl0X3F1ZXVlICpxdWV1ZSwgY29uc3QgY2hhciAqcW5hbWUpDQorew0KKwlf X3JwY19pbml0X3ByaW9yaXR5X3dhaXRfcXVldWUocXVldWUsIHFuYW1lLCAwKTsNCit9DQorRVhQ T1JUX1NZTUJPTChycGNfaW5pdF93YWl0X3F1ZXVlKTsNCisNCiAvKg0KICAqIE1ha2UgYW4gUlBD IHRhc2sgcnVubmFibGUuDQogICoNCkBAIC00MDMsMTcgKzQ4Miw3MiBAQCBycGNfd2FrZV91cF90 YXNrKHN0cnVjdCBycGNfdGFzayAqdGFzaykNCiB9DQogDQogLyoNCisgKiBXYWtlIHVwIHRoZSBu ZXh0IHRhc2sgb24gYSBwcmlvcml0eSBxdWV1ZS4NCisgKi8NCitzdGF0aWMgc3RydWN0IHJwY190 YXNrICogX19ycGNfd2FrZV91cF9uZXh0X3ByaW9yaXR5KHN0cnVjdCBycGNfd2FpdF9xdWV1ZSAq cXVldWUpDQorew0KKwlzdHJ1Y3QgbGlzdF9oZWFkICpxOw0KKwlzdHJ1Y3QgcnBjX3Rhc2sgKnRh c2s7DQorDQorCS8qDQorCSAqIFNlcnZpY2UgYSBiYXRjaCBvZiB0YXNrcyBmcm9tIGEgc2luZ2xl IGNvb2tpZS4NCisJICovDQorCXEgPSAmcXVldWUtPnRhc2tzW3F1ZXVlLT5wcmlvcml0eV07DQor CWlmICghbGlzdF9lbXB0eShxKSkgew0KKwkJdGFzayA9IGxpc3RfZW50cnkocS0+bmV4dCwgc3Ry dWN0IHJwY190YXNrLCB0a19saXN0KTsNCisJCWlmIChxdWV1ZS0+Y29va2llID09IHRhc2stPnRr X2Nvb2tpZSkgew0KKwkJCWlmICgtLXF1ZXVlLT5ucikNCisJCQkJZ290byBvdXQ7DQorCQkJbGlz dF9tb3ZlX3RhaWwoJnRhc2stPnRrX2xpc3QsIHEpOw0KKwkJfQ0KKwkJLyoNCisJCSAqIENoZWNr IGlmIHdlIG5lZWQgdG8gc3dpdGNoIHF1ZXVlcy4NCisJCSAqLw0KKwkJaWYgKC0tcXVldWUtPmNv dW50KQ0KKwkJCWdvdG8gbmV3X2Nvb2tpZTsNCisJfQ0KKw0KKwkvKg0KKwkgKiBTZXJ2aWNlIHRo ZSBuZXh0IHF1ZXVlLg0KKwkgKi8NCisJZG8gew0KKwkJaWYgKHEgPT0gJnF1ZXVlLT50YXNrc1sw XSkNCisJCQlxID0gJnF1ZXVlLT50YXNrc1txdWV1ZS0+bWF4cHJpb3JpdHldOw0KKwkJZWxzZQ0K KwkJCXEgPSBxIC0gMTsNCisJCWlmICghbGlzdF9lbXB0eShxKSkgew0KKwkJCXRhc2sgPSBsaXN0 X2VudHJ5KHEtPm5leHQsIHN0cnVjdCBycGNfdGFzaywgdGtfbGlzdCk7DQorCQkJZ290byBuZXdf cXVldWU7DQorCQl9DQorCX0gd2hpbGUgKHEgIT0gJnF1ZXVlLT50YXNrc1txdWV1ZS0+cHJpb3Jp dHldKTsNCisNCisJcnBjX3Jlc2V0X3dhaXRxdWV1ZV9wcmlvcml0eShxdWV1ZSk7DQorCXJldHVy biBOVUxMOw0KKw0KK25ld19xdWV1ZToNCisJcnBjX3NldF93YWl0cXVldWVfcHJpb3JpdHkocXVl dWUsICh1bnNpZ25lZCBpbnQpKHEgLSAmcXVldWUtPnRhc2tzWzBdKSk7DQorbmV3X2Nvb2tpZToN CisJcnBjX3NldF93YWl0cXVldWVfY29va2llKHF1ZXVlLCB0YXNrLT50a19jb29raWUpOw0KK291 dDoNCisJX19ycGNfd2FrZV91cF90YXNrKHRhc2spOw0KKwlyZXR1cm4gdGFzazsNCit9DQorDQor LyoNCiAgKiBXYWtlIHVwIHRoZSBuZXh0IHRhc2sgb24gdGhlIHdhaXQgcXVldWUuDQogICovDQot c3RydWN0IHJwY190YXNrICoNCi1ycGNfd2FrZV91cF9uZXh0KHN0cnVjdCBycGNfd2FpdF9xdWV1 ZSAqcXVldWUpDQorc3RydWN0IHJwY190YXNrICogcnBjX3dha2VfdXBfbmV4dChzdHJ1Y3QgcnBj X3dhaXRfcXVldWUgKnF1ZXVlKQ0KIHsNCiAJc3RydWN0IHJwY190YXNrCSp0YXNrID0gTlVMTDsN CiANCiAJZHByaW50aygiUlBDOiAgICAgIHdha2VfdXBfbmV4dCglcCBcIiVzXCIpXG4iLCBxdWV1 ZSwgcnBjX3FuYW1lKHF1ZXVlKSk7DQogCXNwaW5fbG9ja19iaCgmcnBjX3F1ZXVlX2xvY2spOw0K LQl0YXNrX2Zvcl9maXJzdCh0YXNrLCAmcXVldWUtPnRhc2tzKQ0KLQkJX19ycGNfd2FrZV91cF90 YXNrKHRhc2spOw0KKwlpZiAoUlBDX0lTX1BSSU9SSVRZKHF1ZXVlKSkNCisJCXRhc2sgPSBfX3Jw Y193YWtlX3VwX25leHRfcHJpb3JpdHkocXVldWUpOw0KKwllbHNlIHsNCisJCXRhc2tfZm9yX2Zp cnN0KHRhc2ssICZxdWV1ZS0+dGFza3NbMF0pDQorCQkJX19ycGNfd2FrZV91cF90YXNrKHRhc2sp Ow0KKwl9DQogCXNwaW5fdW5sb2NrX2JoKCZycGNfcXVldWVfbG9jayk7DQogDQogCXJldHVybiB0 YXNrOw0KQEAgLTQyNSwxNSArNTU5LDIyIEBAIHJwY193YWtlX3VwX25leHQoc3RydWN0IHJwY193 YWl0X3F1ZXVlICoNCiAgKg0KICAqIEdyYWJzIHJwY19xdWV1ZV9sb2NrDQogICovDQotdm9pZA0K LXJwY193YWtlX3VwKHN0cnVjdCBycGNfd2FpdF9xdWV1ZSAqcXVldWUpDQordm9pZCBycGNfd2Fr ZV91cChzdHJ1Y3QgcnBjX3dhaXRfcXVldWUgKnF1ZXVlKQ0KIHsNCiAJc3RydWN0IHJwY190YXNr ICp0YXNrOw0KIA0KKwlzdHJ1Y3QgbGlzdF9oZWFkICpoZWFkOw0KIAlzcGluX2xvY2tfYmgoJnJw Y19xdWV1ZV9sb2NrKTsNCi0Jd2hpbGUgKCFsaXN0X2VtcHR5KCZxdWV1ZS0+dGFza3MpKQ0KLQkJ dGFza19mb3JfZmlyc3QodGFzaywgJnF1ZXVlLT50YXNrcykNCisJaGVhZCA9ICZxdWV1ZS0+dGFz a3NbcXVldWUtPm1heHByaW9yaXR5XTsNCisJZm9yICg7Oykgew0KKwkJd2hpbGUgKCFsaXN0X2Vt cHR5KGhlYWQpKSB7DQorCQkJdGFzayA9IGxpc3RfZW50cnkoaGVhZC0+bmV4dCwgc3RydWN0IHJw Y190YXNrLCB0a19saXN0KTsNCiAJCQlfX3JwY193YWtlX3VwX3Rhc2sodGFzayk7DQorCQl9DQor CQlpZiAoaGVhZCA9PSAmcXVldWUtPnRhc2tzWzBdKQ0KKwkJCWJyZWFrOw0KKwkJaGVhZC0tOw0K Kwl9DQogCXNwaW5fdW5sb2NrX2JoKCZycGNfcXVldWVfbG9jayk7DQogfQ0KIA0KQEAgLTQ0NCwx NyArNTg1LDIyIEBAIHJwY193YWtlX3VwKHN0cnVjdCBycGNfd2FpdF9xdWV1ZSAqcXVldWUNCiAg Kg0KICAqIEdyYWJzIHJwY19xdWV1ZV9sb2NrDQogICovDQotdm9pZA0KLXJwY193YWtlX3VwX3N0 YXR1cyhzdHJ1Y3QgcnBjX3dhaXRfcXVldWUgKnF1ZXVlLCBpbnQgc3RhdHVzKQ0KK3ZvaWQgcnBj X3dha2VfdXBfc3RhdHVzKHN0cnVjdCBycGNfd2FpdF9xdWV1ZSAqcXVldWUsIGludCBzdGF0dXMp DQogew0KKwlzdHJ1Y3QgbGlzdF9oZWFkICpoZWFkOw0KIAlzdHJ1Y3QgcnBjX3Rhc2sgKnRhc2s7 DQogDQogCXNwaW5fbG9ja19iaCgmcnBjX3F1ZXVlX2xvY2spOw0KLQl3aGlsZSAoIWxpc3RfZW1w dHkoJnF1ZXVlLT50YXNrcykpIHsNCi0JCXRhc2tfZm9yX2ZpcnN0KHRhc2ssICZxdWV1ZS0+dGFz a3MpIHsNCisJaGVhZCA9ICZxdWV1ZS0+dGFza3NbcXVldWUtPm1heHByaW9yaXR5XTsNCisJZm9y ICg7Oykgew0KKwkJd2hpbGUgKCFsaXN0X2VtcHR5KGhlYWQpKSB7DQorCQkJdGFzayA9IGxpc3Rf ZW50cnkoaGVhZC0+bmV4dCwgc3RydWN0IHJwY190YXNrLCB0a19saXN0KTsNCiAJCQl0YXNrLT50 a19zdGF0dXMgPSBzdGF0dXM7DQogCQkJX19ycGNfd2FrZV91cF90YXNrKHRhc2spOw0KIAkJfQ0K KwkJaWYgKGhlYWQgPT0gJnF1ZXVlLT50YXNrc1swXSkNCisJCQlicmVhazsNCisJCWhlYWQtLTsN CiAJfQ0KIAlzcGluX3VubG9ja19iaCgmcnBjX3F1ZXVlX2xvY2spOw0KIH0NCkBAIC02NDIsNyAr Nzg4LDcgQEAgX19ycGNfc2NoZWR1bGUodm9pZCkNCiAJd2hpbGUgKDEpIHsNCiAJCXNwaW5fbG9j a19iaCgmcnBjX3F1ZXVlX2xvY2spOw0KIA0KLQkJdGFza19mb3JfZmlyc3QodGFzaywgJnNjaGVk cS50YXNrcykgew0KKwkJdGFza19mb3JfZmlyc3QodGFzaywgJnNjaGVkcS50YXNrc1swXSkgew0K IAkJCV9fcnBjX3JlbW92ZV93YWl0X3F1ZXVlKHRhc2spOw0KIAkJCXNwaW5fdW5sb2NrX2JoKCZy cGNfcXVldWVfbG9jayk7DQogDQpAQCAtNzA2LDkgKzg1Miw3IEBAIHJwY19mcmVlKHN0cnVjdCBy cGNfdGFzayAqdGFzaykNCiAvKg0KICAqIENyZWF0aW9uIGFuZCBkZWxldGlvbiBvZiBSUEMgdGFz ayBzdHJ1Y3R1cmVzDQogICovDQotaW5saW5lIHZvaWQNCi1ycGNfaW5pdF90YXNrKHN0cnVjdCBy cGNfdGFzayAqdGFzaywgc3RydWN0IHJwY19jbG50ICpjbG50LA0KLQkJCQlycGNfYWN0aW9uIGNh bGxiYWNrLCBpbnQgZmxhZ3MpDQordm9pZCBycGNfaW5pdF90YXNrKHN0cnVjdCBycGNfdGFzayAq dGFzaywgc3RydWN0IHJwY19jbG50ICpjbG50LCBycGNfYWN0aW9uIGNhbGxiYWNrLCBpbnQgZmxh Z3MpDQogew0KIAltZW1zZXQodGFzaywgMCwgc2l6ZW9mKCp0YXNrKSk7DQogCWluaXRfdGltZXIo JnRhc2stPnRrX3RpbWVyKTsNCkBAIC03MjYsNiArODcwLDEwIEBAIHJwY19pbml0X3Rhc2soc3Ry dWN0IHJwY190YXNrICp0YXNrLCBzdHINCiAJdGFzay0+dGtfY3JlZF9yZXRyeSA9IDI7DQogCXRh c2stPnRrX3N1aWRfcmV0cnkgPSAxOw0KIA0KKwl0YXNrLT50a19wcmlvcml0eSA9IFJQQ19QUklP UklUWV9OT1JNQUw7DQorCXRhc2stPnRrX2Nvb2tpZSA9ICh1bnNpZ25lZCBsb25nKWN1cnJlbnQ7 DQorCUlOSVRfTElTVF9IRUFEKCZ0YXNrLT50a19saW5rcyk7DQorDQogCS8qIEFkZCB0byBnbG9i YWwgbGlzdCBvZiBhbGwgdGFza3MgKi8NCiAJc3Bpbl9sb2NrKCZycGNfc2NoZWRfbG9jayk7DQog CWxpc3RfYWRkKCZ0YXNrLT50a190YXNrLCAmYWxsX3Rhc2tzKTsNCkBAIC04NjMsNyArMTAxMSw3 IEBAIHJwY19maW5kX3BhcmVudChzdHJ1Y3QgcnBjX3Rhc2sgKmNoaWxkKQ0KIAlzdHJ1Y3QgbGlz dF9oZWFkICpsZTsNCiANCiAJcGFyZW50ID0gKHN0cnVjdCBycGNfdGFzayAqKSBjaGlsZC0+dGtf Y2FsbGRhdGE7DQotCXRhc2tfZm9yX2VhY2godGFzaywgbGUsICZjaGlsZHEudGFza3MpDQorCXRh c2tfZm9yX2VhY2godGFzaywgbGUsICZjaGlsZHEudGFza3NbMF0pDQogCQlpZiAodGFzayA9PSBw YXJlbnQpDQogCQkJcmV0dXJuIHBhcmVudDsNCiANCkBAIC05NDMsNyArMTA5MSw3IEBAIHN0YXRp YyBERUNMQVJFX01VVEVYX0xPQ0tFRChycGNpb2RfcnVubmkNCiBzdGF0aWMgaW5saW5lIGludA0K IHJwY2lvZF90YXNrX3BlbmRpbmcodm9pZCkNCiB7DQotCXJldHVybiAhbGlzdF9lbXB0eSgmc2No ZWRxLnRhc2tzKTsNCisJcmV0dXJuICFsaXN0X2VtcHR5KCZzY2hlZHEudGFza3NbMF0pOw0KIH0N CiANCiANCmRpZmYgLXUgLS1yZWN1cnNpdmUgLS1uZXctZmlsZSAtLXNob3ctYy1mdW5jdGlvbiBs aW51eC0yLjYuNC0yMy11bnJhY2UvbmV0L3N1bnJwYy94cHJ0LmMgbGludXgtMi42LjQtMjQtcnBj X3Rocm90dGxlL25ldC9zdW5ycGMveHBydC5jDQotLS0gbGludXgtMi42LjQtMjMtdW5yYWNlL25l dC9zdW5ycGMveHBydC5jCTIwMDQtMDMtMDQgMTY6NDQ6NTIuMDAwMDAwMDAwIC0wNTAwDQorKysg bGludXgtMi42LjQtMjQtcnBjX3Rocm90dGxlL25ldC9zdW5ycGMveHBydC5jCTIwMDQtMDMtMDQg MTY6NDU6MDIuMDAwMDAwMDAwIC0wNTAwDQpAQCAtMTQ1OCwxMCArMTQ1OCwxMCBAQCB4cHJ0X3Nl dHVwKGludCBwcm90bywgc3RydWN0IHNvY2thZGRyX2luDQogCX0gZWxzZQ0KIAkJeHBydF9kZWZh dWx0X3RpbWVvdXQoJnhwcnQtPnRpbWVvdXQsIHhwcnQtPnByb3QpOw0KIA0KLQlJTklUX1JQQ19X QUlUUSgmeHBydC0+cGVuZGluZywgInhwcnRfcGVuZGluZyIpOw0KLQlJTklUX1JQQ19XQUlUUSgm eHBydC0+c2VuZGluZywgInhwcnRfc2VuZGluZyIpOw0KLQlJTklUX1JQQ19XQUlUUSgmeHBydC0+ cmVzZW5kLCAieHBydF9yZXNlbmQiKTsNCi0JSU5JVF9SUENfV0FJVFEoJnhwcnQtPmJhY2tsb2cs ICJ4cHJ0X2JhY2tsb2ciKTsNCisJcnBjX2luaXRfd2FpdF9xdWV1ZSgmeHBydC0+cGVuZGluZywg InhwcnRfcGVuZGluZyIpOw0KKwlycGNfaW5pdF93YWl0X3F1ZXVlKCZ4cHJ0LT5zZW5kaW5nLCAi eHBydF9zZW5kaW5nIik7DQorCXJwY19pbml0X3dhaXRfcXVldWUoJnhwcnQtPnJlc2VuZCwgInhw cnRfcmVzZW5kIik7DQorCXJwY19pbml0X3ByaW9yaXR5X3dhaXRfcXVldWUoJnhwcnQtPmJhY2ts b2csICJ4cHJ0X2JhY2tsb2ciKTsNCiANCiAJLyogaW5pdGlhbGl6ZSBmcmVlIGxpc3QgKi8NCiAJ Zm9yIChyZXEgPSAmeHBydC0+c2xvdFtlbnRyaWVzLTFdOyByZXEgPj0gJnhwcnQtPnNsb3RbMF07 IHJlcS0tKQ0K --=-88DAsEQoFfsX4cDYh9m3-- --- 25-akpm/fs/nfs/nfs4state.c | 2 25-akpm/fs/nfs/read.c | 1 25-akpm/fs/nfs/write.c | 50 +++++--- 25-akpm/include/linux/nfs_fs.h | 12 + 25-akpm/include/linux/sunrpc/sched.h | 66 ++++++++-- 25-akpm/net/sunrpc/auth_gss/auth_gss.c | 2 25-akpm/net/sunrpc/clnt.c | 2 25-akpm/net/sunrpc/sched.c | 204 ++++++++++++++++++++++++++++----- 25-akpm/net/sunrpc/xprt.c | 8 - 9 files changed, 280 insertions(+), 67 deletions(-) diff -puN fs/nfs/nfs4state.c~nfs-06-rpc_throttle fs/nfs/nfs4state.c --- 25/fs/nfs/nfs4state.c~nfs-06-rpc_throttle 2004-03-23 22:59:29.333597144 -0800 +++ 25-akpm/fs/nfs/nfs4state.c 2004-03-23 22:59:29.353594104 -0800 @@ -105,7 +105,7 @@ nfs4_alloc_client(struct in_addr *addr) INIT_WORK(&clp->cl_renewd, nfs4_renew_state, clp); INIT_LIST_HEAD(&clp->cl_superblocks); init_waitqueue_head(&clp->cl_waitq); - INIT_RPC_WAITQ(&clp->cl_rpcwaitq, "NFS4 client"); + rpc_init_wait_queue(&clp->cl_rpcwaitq, "NFS4 client"); clp->cl_state = 1 << NFS4CLNT_NEW; } return clp; diff -puN fs/nfs/read.c~nfs-06-rpc_throttle fs/nfs/read.c --- 25/fs/nfs/read.c~nfs-06-rpc_throttle 2004-03-23 22:59:29.334596992 -0800 +++ 25-akpm/fs/nfs/read.c 2004-03-23 22:59:29.353594104 -0800 @@ -234,6 +234,7 @@ static void nfs_read_rpcsetup(struct nfs NFS_PROTO(inode)->read_setup(data); + data->task.tk_cookie = (unsigned long)inode; data->task.tk_calldata = data; /* Release requests */ data->task.tk_release = nfs_readdata_release; diff -puN fs/nfs/write.c~nfs-06-rpc_throttle fs/nfs/write.c --- 25/fs/nfs/write.c~nfs-06-rpc_throttle 2004-03-23 22:59:29.336596688 -0800 +++ 25-akpm/fs/nfs/write.c 2004-03-23 22:59:29.355593800 -0800 @@ -173,15 +173,14 @@ static void nfs_mark_uptodate(struct pag * Write a page synchronously. * Offset is the data offset within the page. */ -static int -nfs_writepage_sync(struct file *file, struct inode *inode, struct page *page, - unsigned int offset, unsigned int count) +static int nfs_writepage_sync(struct file *file, struct inode *inode, + struct page *page, unsigned int offset, unsigned int count, + int how) { unsigned int wsize = NFS_SERVER(inode)->wsize; int result, written = 0; - int swapfile = IS_SWAPFILE(inode); struct nfs_write_data wdata = { - .flags = swapfile ? NFS_RPC_SWAPFLAGS : 0, + .flags = how, .cred = NULL, .inode = inode, .args = { @@ -205,7 +204,7 @@ nfs_writepage_sync(struct file *file, st nfs_begin_data_update(inode); do { - if (count < wsize && !swapfile) + if (count < wsize) wdata.args.count = count; wdata.args.offset = page_offset(page) + wdata.args.pgbase; @@ -260,6 +259,15 @@ static int nfs_writepage_async(struct fi return status; } +static int wb_priority(struct writeback_control *wbc) +{ + if (wbc->for_reclaim) + return FLUSH_HIGHPRI; + if (wbc->for_kupdate) + return FLUSH_LOWPRI; + return 0; +} + /* * Write an mmapped page to the server. */ @@ -270,6 +278,7 @@ int nfs_writepage(struct page *page, str unsigned offset = PAGE_CACHE_SIZE; loff_t i_size = i_size_read(inode); int inode_referenced = 0; + int priority = wb_priority(wbc); int err; /* @@ -285,7 +294,7 @@ int nfs_writepage(struct page *page, str end_index = i_size >> PAGE_CACHE_SHIFT; /* Ensure we've flushed out any previous writes */ - nfs_wb_page(inode,page); + nfs_wb_page_priority(inode, page, priority); /* easy case */ if (page->index < end_index) @@ -307,7 +316,7 @@ do_it: err = WRITEPAGE_ACTIVATE; } } else { - err = nfs_writepage_sync(NULL, inode, page, 0, offset); + err = nfs_writepage_sync(NULL, inode, page, 0, offset, priority); if (err == offset) err = 0; } @@ -338,7 +347,7 @@ int nfs_writepages(struct address_space return 0; nfs_wait_on_write_congestion(mapping, 0); } - err = nfs_flush_inode(inode, 0, 0, 0); + err = nfs_flush_inode(inode, 0, 0, wb_priority(wbc)); if (err < 0) goto out; wbc->nr_to_write -= err; @@ -347,7 +356,7 @@ int nfs_writepages(struct address_space if (err < 0) goto out; } - err = nfs_commit_inode(inode, 0, 0, 0); + err = nfs_commit_inode(inode, 0, 0, wb_priority(wbc)); if (err > 0) wbc->nr_to_write -= err; out: @@ -719,8 +728,8 @@ nfs_flush_incompatible(struct file *file * XXX: Keep an eye on generic_file_read to make sure it doesn't do bad * things with a page scheduled for an RPC call (e.g. invalidate it). */ -int -nfs_updatepage(struct file *file, struct page *page, unsigned int offset, unsigned int count) +int nfs_updatepage(struct file *file, struct page *page, + unsigned int offset, unsigned int count) { struct dentry *dentry = file->f_dentry; struct inode *inode = page->mapping->host; @@ -732,7 +741,7 @@ nfs_updatepage(struct file *file, struct count, (long long)(page_offset(page) +offset)); if (IS_SYNC(inode)) { - status = nfs_writepage_sync(file, inode, page, offset, count); + status = nfs_writepage_sync(file, inode, page, offset, count, 0); if (status > 0) { if (offset == 0 && status == PAGE_CACHE_SIZE) SetPageUptodate(page); @@ -819,6 +828,17 @@ out: nfs_unlock_request(req); } +static inline int flush_task_priority(int how) +{ + switch (how & (FLUSH_HIGHPRI|FLUSH_LOWPRI)) { + case FLUSH_HIGHPRI: + return RPC_PRIORITY_HIGH; + case FLUSH_LOWPRI: + return RPC_PRIORITY_LOW; + } + return RPC_PRIORITY_NORMAL; +} + /* * Set up the argument/result storage required for the RPC call. */ @@ -848,6 +868,8 @@ static void nfs_write_rpcsetup(struct nf NFS_PROTO(inode)->write_setup(data, how); + data->task.tk_priority = flush_task_priority(how); + data->task.tk_cookie = (unsigned long)inode; data->task.tk_calldata = data; /* Release requests */ data->task.tk_release = nfs_writedata_release; @@ -1210,6 +1232,8 @@ static void nfs_commit_rpcsetup(struct l NFS_PROTO(inode)->commit_setup(data, how); + data->task.tk_priority = flush_task_priority(how); + data->task.tk_cookie = (unsigned long)inode; data->task.tk_calldata = data; /* Release requests */ data->task.tk_release = nfs_commit_release; diff -puN include/linux/nfs_fs.h~nfs-06-rpc_throttle include/linux/nfs_fs.h --- 25/include/linux/nfs_fs.h~nfs-06-rpc_throttle 2004-03-23 22:59:29.338596384 -0800 +++ 25-akpm/include/linux/nfs_fs.h 2004-03-23 22:59:29.356593648 -0800 @@ -69,6 +69,8 @@ #define FLUSH_SYNC 1 /* file being synced, or contention */ #define FLUSH_WAIT 2 /* wait for completion */ #define FLUSH_STABLE 4 /* commit to stable storage */ +#define FLUSH_LOWPRI 8 /* low priority background flush */ +#define FLUSH_HIGHPRI 16 /* high priority memory reclaim flush */ #ifdef __KERNEL__ @@ -374,14 +376,18 @@ nfs_wb_all(struct inode *inode) /* * Write back all requests on one page - we do this before reading it. */ -static inline int -nfs_wb_page(struct inode *inode, struct page* page) +static inline int nfs_wb_page_priority(struct inode *inode, struct page* page, int how) { int error = nfs_sync_inode(inode, page->index, 1, - FLUSH_WAIT | FLUSH_STABLE); + how | FLUSH_WAIT | FLUSH_STABLE); return (error < 0) ? error : 0; } +static inline int nfs_wb_page(struct inode *inode, struct page* page) +{ + return nfs_wb_page_priority(inode, page, 0); +} + /* Hack for future NFS swap support */ #ifndef IS_SWAPFILE # define IS_SWAPFILE(inode) (0) diff -puN include/linux/sunrpc/sched.h~nfs-06-rpc_throttle include/linux/sunrpc/sched.h --- 25/include/linux/sunrpc/sched.h~nfs-06-rpc_throttle 2004-03-23 22:59:29.339596232 -0800 +++ 25-akpm/include/linux/sunrpc/sched.h 2004-03-23 22:59:29.357593496 -0800 @@ -49,6 +49,8 @@ struct rpc_task { tk_cred_retry, tk_suid_retry; + unsigned long tk_cookie; /* Cookie for batching tasks */ + /* * timeout_fn to be executed by timer bottom half * callback to be executed after waking up @@ -72,7 +74,9 @@ struct rpc_task { unsigned long tk_timeout; /* timeout for rpc_sleep() */ unsigned short tk_flags; /* misc flags */ unsigned char tk_active : 1;/* Task has been activated */ + unsigned char tk_priority : 2;/* Task priority */ unsigned long tk_runstate; /* Task run status */ + struct list_head tk_links; /* links to related tasks */ #ifdef RPC_DEBUG unsigned short tk_pid; /* debugging aid */ #endif @@ -138,28 +142,58 @@ typedef void (*rpc_action)(struct rpc_ } while(0) /* + * Task priorities. + * Note: if you change these, you must also change + * the task initialization definitions below. + */ +#define RPC_PRIORITY_LOW 0 +#define RPC_PRIORITY_NORMAL 1 +#define RPC_PRIORITY_HIGH 2 +#define RPC_NR_PRIORITY (RPC_PRIORITY_HIGH+1) + +/* * RPC synchronization objects */ struct rpc_wait_queue { - struct list_head tasks; + struct list_head tasks[RPC_NR_PRIORITY]; /* task queue for each priority level */ + unsigned long cookie; /* cookie of last task serviced */ + unsigned char maxpriority; /* maximum priority (0 if queue is not a priority queue) */ + unsigned char priority; /* current priority */ + unsigned char count; /* # task groups remaining serviced so far */ + unsigned char nr; /* # tasks remaining for cookie */ #ifdef RPC_DEBUG - char * name; + const char * name; #endif }; +/* + * This is the # requests to send consecutively + * from a single cookie. The aim is to improve + * performance of NFS operations such as read/write. + */ +#define RPC_BATCH_COUNT 16 + #ifndef RPC_DEBUG -# define RPC_WAITQ_INIT(var,qname) ((struct rpc_wait_queue) {LIST_HEAD_INIT(var)}) -# define RPC_WAITQ(var,qname) struct rpc_wait_queue var = RPC_WAITQ_INIT(var.tasks,qname) -# define INIT_RPC_WAITQ(ptr,qname) do { \ - INIT_LIST_HEAD(&(ptr)->tasks); \ - } while(0) +# define RPC_WAITQ_INIT(var,qname) { \ + .tasks = { \ + [0] = LIST_HEAD_INIT(var.tasks[0]), \ + [1] = LIST_HEAD_INIT(var.tasks[1]), \ + [2] = LIST_HEAD_INIT(var.tasks[2]), \ + }, \ + } #else -# define RPC_WAITQ_INIT(var,qname) ((struct rpc_wait_queue) {LIST_HEAD_INIT(var.tasks), qname}) -# define RPC_WAITQ(var,qname) struct rpc_wait_queue var = RPC_WAITQ_INIT(var,qname) -# define INIT_RPC_WAITQ(ptr,qname) do { \ - INIT_LIST_HEAD(&(ptr)->tasks); (ptr)->name = qname; \ - } while(0) +# define RPC_WAITQ_INIT(var,qname) { \ + .tasks = { \ + [0] = LIST_HEAD_INIT(var.tasks[0]), \ + [1] = LIST_HEAD_INIT(var.tasks[1]), \ + [2] = LIST_HEAD_INIT(var.tasks[2]), \ + }, \ + .name = qname, \ + } #endif +# define RPC_WAITQ(var,qname) struct rpc_wait_queue var = RPC_WAITQ_INIT(var,qname) + +#define RPC_IS_PRIORITY(q) ((q)->maxpriority > 0) /* * Function prototypes @@ -175,6 +209,8 @@ void rpc_run_child(struct rpc_task *par rpc_action action); int rpc_add_wait_queue(struct rpc_wait_queue *, struct rpc_task *); void rpc_remove_wait_queue(struct rpc_task *); +void rpc_init_priority_wait_queue(struct rpc_wait_queue *, const char *); +void rpc_init_wait_queue(struct rpc_wait_queue *, const char *); void rpc_sleep_on(struct rpc_wait_queue *, struct rpc_task *, rpc_action action, rpc_action timer); void rpc_add_timer(struct rpc_task *, rpc_action); @@ -194,16 +230,14 @@ void rpc_show_tasks(void); int rpc_init_mempool(void); void rpc_destroy_mempool(void); -static __inline__ void -rpc_exit(struct rpc_task *task, int status) +static inline void rpc_exit(struct rpc_task *task, int status) { task->tk_status = status; task->tk_action = NULL; } #ifdef RPC_DEBUG -static __inline__ char * -rpc_qname(struct rpc_wait_queue *q) +static inline const char * rpc_qname(struct rpc_wait_queue *q) { return ((q && q->name) ? q->name : "unknown"); } diff -puN net/sunrpc/auth_gss/auth_gss.c~nfs-06-rpc_throttle net/sunrpc/auth_gss/auth_gss.c --- 25/net/sunrpc/auth_gss/auth_gss.c~nfs-06-rpc_throttle 2004-03-23 22:59:29.341595928 -0800 +++ 25-akpm/net/sunrpc/auth_gss/auth_gss.c 2004-03-23 22:59:29.358593344 -0800 @@ -365,7 +365,7 @@ retry: gss_msg = gss_new; memset(gss_new, 0, sizeof(*gss_new)); INIT_LIST_HEAD(&gss_new->list); - INIT_RPC_WAITQ(&gss_new->waitq, "RPCSEC_GSS upcall waitq"); + rpc_init_wait_queue(&gss_new->waitq, "RPCSEC_GSS upcall waitq"); atomic_set(&gss_new->count, 2); msg = &gss_new->msg; msg->data = &gss_new->uid; diff -puN net/sunrpc/clnt.c~nfs-06-rpc_throttle net/sunrpc/clnt.c --- 25/net/sunrpc/clnt.c~nfs-06-rpc_throttle 2004-03-23 22:59:29.343595624 -0800 +++ 25-akpm/net/sunrpc/clnt.c 2004-03-23 22:59:29.359593192 -0800 @@ -144,7 +144,7 @@ rpc_create_client(struct rpc_xprt *xprt, clnt->cl_vers = version->number; clnt->cl_prot = xprt->prot; clnt->cl_stats = program->stats; - INIT_RPC_WAITQ(&clnt->cl_pmap_default.pm_bindwait, "bindwait"); + rpc_init_wait_queue(&clnt->cl_pmap_default.pm_bindwait, "bindwait"); if (!clnt->cl_port) clnt->cl_autobind = 1; diff -puN net/sunrpc/sched.c~nfs-06-rpc_throttle net/sunrpc/sched.c --- 25/net/sunrpc/sched.c~nfs-06-rpc_throttle 2004-03-23 22:59:29.344595472 -0800 +++ 25-akpm/net/sunrpc/sched.c 2004-03-23 22:59:29.361592888 -0800 @@ -162,6 +162,26 @@ rpc_delete_timer(struct rpc_task *task) } /* + * Add new request to a priority queue. + */ +static void __rpc_add_wait_queue_priority(struct rpc_wait_queue *queue, struct rpc_task *task) +{ + struct list_head *q; + struct rpc_task *t; + + q = &queue->tasks[task->tk_priority]; + if (unlikely(task->tk_priority > queue->maxpriority)) + q = &queue->tasks[queue->maxpriority]; + list_for_each_entry(t, q, tk_list) { + if (t->tk_cookie == task->tk_cookie) { + list_add_tail(&task->tk_list, &t->tk_links); + return; + } + } + list_add_tail(&task->tk_list, q); +} + +/* * Add new request to wait queue. * * Swapper tasks always get inserted at the head of the queue. @@ -169,8 +189,7 @@ rpc_delete_timer(struct rpc_task *task) * improve overall performance. * Everyone else gets appended to the queue to ensure proper FIFO behavior. */ -static inline int -__rpc_add_wait_queue(struct rpc_wait_queue *queue, struct rpc_task *task) +static int __rpc_add_wait_queue(struct rpc_wait_queue *queue, struct rpc_task *task) { if (task->tk_rpcwait == queue) return 0; @@ -179,10 +198,12 @@ __rpc_add_wait_queue(struct rpc_wait_que printk(KERN_WARNING "RPC: doubly enqueued task!\n"); return -EWOULDBLOCK; } - if (RPC_IS_SWAPPER(task)) - list_add(&task->tk_list, &queue->tasks); + if (RPC_IS_PRIORITY(queue)) + __rpc_add_wait_queue_priority(queue, task); + else if (RPC_IS_SWAPPER(task)) + list_add(&task->tk_list, &queue->tasks[0]); else - list_add_tail(&task->tk_list, &queue->tasks); + list_add_tail(&task->tk_list, &queue->tasks[0]); task->tk_rpcwait = queue; dprintk("RPC: %4d added to queue %p \"%s\"\n", @@ -191,8 +212,7 @@ __rpc_add_wait_queue(struct rpc_wait_que return 0; } -int -rpc_add_wait_queue(struct rpc_wait_queue *q, struct rpc_task *task) +int rpc_add_wait_queue(struct rpc_wait_queue *q, struct rpc_task *task) { int result; @@ -203,18 +223,35 @@ rpc_add_wait_queue(struct rpc_wait_queue } /* + * Remove request from a priority queue. + */ +static void __rpc_remove_wait_queue_priority(struct rpc_task *task) +{ + struct rpc_task *t; + + if (!list_empty(&task->tk_links)) { + t = list_entry(task->tk_links.next, struct rpc_task, tk_list); + list_move(&t->tk_list, &task->tk_list); + list_splice_init(&task->tk_links, &t->tk_links); + } + list_del(&task->tk_list); +} + +/* * Remove request from queue. * Note: must be called with spin lock held. */ -static inline void -__rpc_remove_wait_queue(struct rpc_task *task) +static void __rpc_remove_wait_queue(struct rpc_task *task) { struct rpc_wait_queue *queue = task->tk_rpcwait; if (!queue) return; - list_del(&task->tk_list); + if (RPC_IS_PRIORITY(queue)) + __rpc_remove_wait_queue_priority(task); + else + list_del(&task->tk_list); task->tk_rpcwait = NULL; dprintk("RPC: %4d removed from queue %p \"%s\"\n", @@ -231,6 +268,48 @@ rpc_remove_wait_queue(struct rpc_task *t spin_unlock_bh(&rpc_queue_lock); } +static inline void rpc_set_waitqueue_priority(struct rpc_wait_queue *queue, int priority) +{ + queue->priority = priority; + queue->count = 1 << (priority * 2); +} + +static inline void rpc_set_waitqueue_cookie(struct rpc_wait_queue *queue, unsigned long cookie) +{ + queue->cookie = cookie; + queue->nr = RPC_BATCH_COUNT; +} + +static inline void rpc_reset_waitqueue_priority(struct rpc_wait_queue *queue) +{ + rpc_set_waitqueue_priority(queue, queue->maxpriority); + rpc_set_waitqueue_cookie(queue, 0); +} + +static void __rpc_init_priority_wait_queue(struct rpc_wait_queue *queue, const char *qname, int maxprio) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(queue->tasks); i++) + INIT_LIST_HEAD(&queue->tasks[i]); + queue->maxpriority = maxprio; + rpc_reset_waitqueue_priority(queue); +#ifdef RPC_DEBUG + queue->name = qname; +#endif +} + +void rpc_init_priority_wait_queue(struct rpc_wait_queue *queue, const char *qname) +{ + __rpc_init_priority_wait_queue(queue, qname, RPC_PRIORITY_HIGH); +} + +void rpc_init_wait_queue(struct rpc_wait_queue *queue, const char *qname) +{ + __rpc_init_priority_wait_queue(queue, qname, 0); +} +EXPORT_SYMBOL(rpc_init_wait_queue); + /* * Make an RPC task runnable. * @@ -403,17 +482,72 @@ rpc_wake_up_task(struct rpc_task *task) } /* + * Wake up the next task on a priority queue. + */ +static struct rpc_task * __rpc_wake_up_next_priority(struct rpc_wait_queue *queue) +{ + struct list_head *q; + struct rpc_task *task; + + /* + * Service a batch of tasks from a single cookie. + */ + q = &queue->tasks[queue->priority]; + if (!list_empty(q)) { + task = list_entry(q->next, struct rpc_task, tk_list); + if (queue->cookie == task->tk_cookie) { + if (--queue->nr) + goto out; + list_move_tail(&task->tk_list, q); + } + /* + * Check if we need to switch queues. + */ + if (--queue->count) + goto new_cookie; + } + + /* + * Service the next queue. + */ + do { + if (q == &queue->tasks[0]) + q = &queue->tasks[queue->maxpriority]; + else + q = q - 1; + if (!list_empty(q)) { + task = list_entry(q->next, struct rpc_task, tk_list); + goto new_queue; + } + } while (q != &queue->tasks[queue->priority]); + + rpc_reset_waitqueue_priority(queue); + return NULL; + +new_queue: + rpc_set_waitqueue_priority(queue, (unsigned int)(q - &queue->tasks[0])); +new_cookie: + rpc_set_waitqueue_cookie(queue, task->tk_cookie); +out: + __rpc_wake_up_task(task); + return task; +} + +/* * Wake up the next task on the wait queue. */ -struct rpc_task * -rpc_wake_up_next(struct rpc_wait_queue *queue) +struct rpc_task * rpc_wake_up_next(struct rpc_wait_queue *queue) { struct rpc_task *task = NULL; dprintk("RPC: wake_up_next(%p \"%s\")\n", queue, rpc_qname(queue)); spin_lock_bh(&rpc_queue_lock); - task_for_first(task, &queue->tasks) - __rpc_wake_up_task(task); + if (RPC_IS_PRIORITY(queue)) + task = __rpc_wake_up_next_priority(queue); + else { + task_for_first(task, &queue->tasks[0]) + __rpc_wake_up_task(task); + } spin_unlock_bh(&rpc_queue_lock); return task; @@ -425,15 +559,22 @@ rpc_wake_up_next(struct rpc_wait_queue * * * Grabs rpc_queue_lock */ -void -rpc_wake_up(struct rpc_wait_queue *queue) +void rpc_wake_up(struct rpc_wait_queue *queue) { struct rpc_task *task; + struct list_head *head; spin_lock_bh(&rpc_queue_lock); - while (!list_empty(&queue->tasks)) - task_for_first(task, &queue->tasks) + head = &queue->tasks[queue->maxpriority]; + for (;;) { + while (!list_empty(head)) { + task = list_entry(head->next, struct rpc_task, tk_list); __rpc_wake_up_task(task); + } + if (head == &queue->tasks[0]) + break; + head--; + } spin_unlock_bh(&rpc_queue_lock); } @@ -444,17 +585,22 @@ rpc_wake_up(struct rpc_wait_queue *queue * * Grabs rpc_queue_lock */ -void -rpc_wake_up_status(struct rpc_wait_queue *queue, int status) +void rpc_wake_up_status(struct rpc_wait_queue *queue, int status) { + struct list_head *head; struct rpc_task *task; spin_lock_bh(&rpc_queue_lock); - while (!list_empty(&queue->tasks)) { - task_for_first(task, &queue->tasks) { + head = &queue->tasks[queue->maxpriority]; + for (;;) { + while (!list_empty(head)) { + task = list_entry(head->next, struct rpc_task, tk_list); task->tk_status = status; __rpc_wake_up_task(task); } + if (head == &queue->tasks[0]) + break; + head--; } spin_unlock_bh(&rpc_queue_lock); } @@ -642,7 +788,7 @@ __rpc_schedule(void) while (1) { spin_lock_bh(&rpc_queue_lock); - task_for_first(task, &schedq.tasks) { + task_for_first(task, &schedq.tasks[0]) { __rpc_remove_wait_queue(task); spin_unlock_bh(&rpc_queue_lock); @@ -706,9 +852,7 @@ rpc_free(struct rpc_task *task) /* * Creation and deletion of RPC task structures */ -inline void -rpc_init_task(struct rpc_task *task, struct rpc_clnt *clnt, - rpc_action callback, int flags) +void rpc_init_task(struct rpc_task *task, struct rpc_clnt *clnt, rpc_action callback, int flags) { memset(task, 0, sizeof(*task)); init_timer(&task->tk_timer); @@ -726,6 +870,10 @@ rpc_init_task(struct rpc_task *task, str task->tk_cred_retry = 2; task->tk_suid_retry = 1; + task->tk_priority = RPC_PRIORITY_NORMAL; + task->tk_cookie = (unsigned long)current; + INIT_LIST_HEAD(&task->tk_links); + /* Add to global list of all tasks */ spin_lock(&rpc_sched_lock); list_add(&task->tk_task, &all_tasks); @@ -863,7 +1011,7 @@ rpc_find_parent(struct rpc_task *child) struct list_head *le; parent = (struct rpc_task *) child->tk_calldata; - task_for_each(task, le, &childq.tasks) + task_for_each(task, le, &childq.tasks[0]) if (task == parent) return parent; @@ -943,7 +1091,7 @@ static DECLARE_MUTEX_LOCKED(rpciod_runni static inline int rpciod_task_pending(void) { - return !list_empty(&schedq.tasks); + return !list_empty(&schedq.tasks[0]); } diff -puN net/sunrpc/xprt.c~nfs-06-rpc_throttle net/sunrpc/xprt.c --- 25/net/sunrpc/xprt.c~nfs-06-rpc_throttle 2004-03-23 22:59:29.345595320 -0800 +++ 25-akpm/net/sunrpc/xprt.c 2004-03-23 22:59:29.362592736 -0800 @@ -1458,10 +1458,10 @@ xprt_setup(int proto, struct sockaddr_in } else xprt_default_timeout(&xprt->timeout, xprt->prot); - INIT_RPC_WAITQ(&xprt->pending, "xprt_pending"); - INIT_RPC_WAITQ(&xprt->sending, "xprt_sending"); - INIT_RPC_WAITQ(&xprt->resend, "xprt_resend"); - INIT_RPC_WAITQ(&xprt->backlog, "xprt_backlog"); + rpc_init_wait_queue(&xprt->pending, "xprt_pending"); + rpc_init_wait_queue(&xprt->sending, "xprt_sending"); + rpc_init_wait_queue(&xprt->resend, "xprt_resend"); + rpc_init_priority_wait_queue(&xprt->backlog, "xprt_backlog"); /* initialize free list */ for (req = &xprt->slot[entries-1]; req >= &xprt->slot[0]; req--) _